Race condition in ldif2ldbm (ITS#140)

[dave@chello.com]

Full_Name: David Boyce
Version: 1.2 stable
OS: Solaris 2.6
URL: 
Submission from: (NULL) (212.83.65.232)


If ldif2ldbm is run with a number of jobs, there is a potential race
condition. It only seems to manifest itself when there are a large number
of entries, eg, something between 10000 and 100000.

The problem is the NEXTID file. If no extra jobs are run, then the NEXTID file
is written by ldif2id2entry before any other processes are started. Each
subsequent
job will read a correct NEXTID file, and write out the same value.
But if a number of jobs can run in parallel, then it is possible for the NEXTID
file to get into an inconsistent state. ldif2id2entry doesn't write out the
file until it finishes processing, and if another job completes before it,
then the NEXTID file will contain the value '1'. If a new job starts before
ldif2id2entry completes, then it will read the value '1', and write that
value on termination.

For instance, here is a log:

# ldif2ldbm -i 100k.ldif -f /usr/local/openldap/etc/openldap/100k.conf -j 4
Written 1 to NEXTID file
Read 1 from NEXTID file
ldif2id2entry: written 100008 to NEXTID file
Written 1 to NEXTID file
Written 1 to NEXTID file
Read 1 from NEXTID file
Written 1 to NEXTID file

The NEXTID file contains the next entry ID to be allocated. If this is wrong,
then lots of bad things can happen. For instance, entries can get overwritten,
sub-tree searches at root stop working, etc.

The solution is to ensure that ldif2id2entry completes before any other job
starts. I've appended a quick-and-dirty patch:

*** ldif2ldbm.c~	Thu Jan 14 02:57:52 1999
--- ldif2ldbm.c	Tue Apr 27 10:52:44 1999
***************
*** 20,26 ****
  #define ID2CHILDRENCMD		"ldif2id2children"
  #define MAXARGS      		100
  
! static void fork_child( char *prog, char *args[] );
  static void	wait4kids( int nkidval );
  
  static char	*indexcmd;
--- 20,26 ----
  #define ID2CHILDRENCMD		"ldif2id2children"
  #define MAXARGS      		100
  
! static void fork_child( char *prog, char *args[], int wait );
  static void	wait4kids( int nkidval );
  
  static char	*indexcmd;
***************
*** 151,157 ****
  		args[i++] = buf3;
  	}
  	args[i++] = NULL;
! 	fork_child( cmd, args );
  
  	/*
  	 * generate the dn2id and id2children indexes
--- 151,157 ----
  		args[i++] = buf3;
  	}
  	args[i++] = NULL;
! 	fork_child( cmd, args, 0 );
  
  	/*
  	 * generate the dn2id and id2children indexes
***************
*** 173,179 ****
  		args[i++] = buf3;
  	}
  	args[i++] = NULL;
! 	fork_child( cmd, args );
  
  	/*
  	 * generate the attribute indexes
--- 173,179 ----
  		args[i++] = buf3;
  	}
  	args[i++] = NULL;
! 	fork_child( cmd, args, 1 );  /* Wait for id2entry to complete first. */
  
  	/*
  	 * generate the attribute indexes
***************
*** 252,258 ****
  					    &indexmask, &syntaxmask );
  					if ( indexmask ) {
  						args[i - 2] = type;
! 						fork_child( cmd, args );
  					}
  				}
  			}
--- 252,258 ----
  					    &indexmask, &syntaxmask );
  					if ( indexmask ) {
  						args[i - 2] = type;
! 						fork_child( cmd, args, 0 );
  					}
  				}
  			}
***************
*** 268,278 ****
  }
  
  static void
! fork_child( char *prog, char *args[] )
  {
  	int	status, pid;
  
! 	wait4kids( maxkids );
  
  	switch ( pid = fork() ) {
  	case 0:		/* child */
--- 268,281 ----
  }
  
  static void
! fork_child( char *prog, char *args[], int wait )
  {
  	int	status, pid;
  
! 	if (wait)
! 	  wait4kids (-1);
! 	else
! 	  wait4kids( maxkids );
  
  	switch ( pid = fork() ) {
  	case 0:		/* child */