[Date Prev][Date Next] [Chronological] [Thread] [Top]
A couple of slapd/slurpd replication suggestions.

To: openldap-devel@OpenLDAP.org
Subject: A couple of slapd/slurpd replication suggestions.
From: Stephen Rush <Stephen.Rush@liffe.com>
Date: Fri, 15 Jan 1999 15:11:26 -0000
Hi, 

After a couple of weeks configuring/playing around with slapd and slurpd I
have a couple of suggestions: 

Firstly: Whilst writing some scripts to generate slapd configuration files
for a number of machines I was quite surprised to find that slurpd does not
read include files. Hence, if you have a configuration file of the form:

	# Main Configuration File 
	database ldbm
	#    :
	#    :
	include    /usr/local/LDAP/slapd_replication_config
	#    :

together with a /usr/local/LDAP/slapd_replication_config file 

		# Definition of replication details
		replica	host=xxx .......

Then slapd will read the replication configuration file but slurpd will not,
hence slurpd will complain that there are no replicas specified in the
config file and will abort. The changes to allow slurpd to read include
files appear pretty simple and are based on the slapd configuration routines
(see below).
Is there any reason for not allowing this to be done? 

Secondly: Replication in LDAP has an obvious master slave relationship
between a master who stores the data and a number of slaves which can be
used for read access to it. slurpd allows the master to replicate data to
the slaves, but there appears to be no way for slaves to replicate data back
to the master if a client performs an update on one of the slaves rather
than on the master. (If they use slurpd then the updates end up in an
infinite cycle).
However, for clients to use this model in write mode they must always be
aware of which server is the master and which ones are not (otherwise their
updates will not be replicated to the other servers). In our application at
least, this seems to be a rather unnecessary restriction. 
Whilst I'm sure the slaves could generate a referral and it could all be
made to work, this seems rather a complicated way to do things. 
Instead, slurpd/slapd currently have a configuration parameter which
indicates the dn to use when performing replication. Would it not be
reasonable to have a further configuration item which indicates a dn for a
user who can perform updates to the database, but who's updates are not
themselves replicated. 
Again, the changes are not large (though they touch a few more files, see
below) and perhaps other people may also find them useful. Are there any
pressing reasons why this kind of functionality should not be allowed? 
If this were allowed then the following configuration files would allow two
slapd servers to replicate changes to each other. 

--------------- SLAPD_CONFIG_HOST_1 -------------------------------------
# SLAPD CONFIG FILE FOR HOST1 - REPLICTES TO HOST2
# The backup database will replicate to us as this user, do NOT replicate 
# these changes on to anyone else or we get in a loop. 
updatenorepdn	"cn=replicator2,o=organisation,c=UK"
	
# The root user can access anything, as can the replication users. 
access to *	by "dn=cn=admin,o=organisation,c=UK" write 
				by "dn=cn=replicator2,o=organisation,c=UK"
write 
	
# Nobody else needs to be able to do anything at all.
defaultaccess 	none

replica		host=host2
				"binddn=cn=replicator1,o=organisation,c=UK"
				bindmethod=simple
				credentials=password
# END

--------------- SLAPD_CONFIG_HOST_2 -------------------------------------
# SLAPD CONFIG FILE FOR HOST2 - REPLICTES TO HOST1
# The backup database will replicate to us as this user, do NOT replicate 
# these changes on to anyone else or we get in a loop. 
updatenorepdn	"cn=replicator1,o=organisation,c=UK"

# The root user can access anything, as can the replication users. 
access to *	by "dn=cn=admin,o=organisation,c=UK" write 
				by "dn=cn=replicator1,o=organisation,c=UK"
write 
	
	# Nobody else needs to be able to do anything at all.
defaultaccess 	none

replica		host=host1
				"binddn=cn=replicator2,o=organisation,c=UK"
				bindmethod=simple
				credentials=password
# END


Any comments on either of these suggestions ? 

Steve R


1)	Changes to allow slurpd to read nested include files:

	*** ldap/servers/slurpd/config.c	Fri Jan 15 14:27:30 1999
	--- ldap/servers/slurpd/config.c.save	Fri Jan 15 14:27:20 1999
	***************
	*** 56,61 ****
	--- 56,62 ----
	  {
	      FILE	*fp;
	      char	*line;
	+     char	*savefname;
	      int		cargc;
	      char	*cargv[MAXARGS];
	  
	***************
	*** 107,112 ****
	--- 108,123 ----
	  	    }
	  	} else if ( strcasecmp( cargv[0], "replica" ) == 0 ) {
	  	    add_replica( cargv, cargc );
	+ 	} else if ( strcasecmp( cargv[0], "include" ) == 0 ) {
	+ 		if ( cargc < 2 ) {
	+ 			Debug( LDAP_DEBUG_ANY,
	+ 			       "%s: missing filename in \"include
<filename>\" line\n",
	+ 			       fname, 0, 0 );
	+ 			exit( 1 );
	+ 		}
	+ 		savefname = strdup( cargv[1] );
	+ 		slurpd_read_config( savefname);
	+ 		free( savefname );
	  	}
	      }
	      fclose( fp );

2)	Changes to allow replication from slaves back to the master server.

*** ldap/servers/slapd/add.c	Fri Jan 15 15:00:14 1999
--- ldap/servers/slapd/add.c.save	Fri Jan 15 14:57:37 1999
***************
*** 119,125 ****
  	if ( be->be_add != NULL ) {
  		/* do the update here */
  		if ( be->be_updatedn == NULL ||
! 			strcasecmp( be->be_updatedn, op->o_dn ) == 0 ) {
  
  			if ( (be->be_lastmod == ON || (be->be_lastmod ==
UNDEFINED &&
  				global_lastmod == ON)) && be->be_updatedn ==
NULL ) {
--- 119,127 ----
  	if ( be->be_add != NULL ) {
  		/* do the update here */
  		if ( be->be_updatedn == NULL ||
! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
! 		     (be->be_updatenorepdn && 
! 		      strcasecmp( be->be_updatenorepdn, op->o_dn ) == 0 )) {
  
  			if ( (be->be_lastmod == ON || (be->be_lastmod ==
UNDEFINED &&
  				global_lastmod == ON)) && be->be_updatedn ==
NULL ) {
***************
*** 127,133 ****
  				add_created_attrs( op, e );
  			}
  			if ( (*be->be_add)( be, conn, op, e ) == 0 ) {
! 				replog( be, LDAP_REQ_ADD, e->e_dn, e, 0 );
  			}
  
  		} else {
--- 129,138 ----
  				add_created_attrs( op, e );
  			}
  			if ( (*be->be_add)( be, conn, op, e ) == 0 ) {
! 			  	if (!be->be_updatenorepdn ||
! 				    strcasecmp( be->be_updatenorepdn,
op->o_dn ) != 0) {
! 				  	replog( be, LDAP_REQ_ADD, e->e_dn,
e, 0 );
! 				}
  			}
  
  		} else {
*** ldap/servers/slapd/config.c	Fri Jan 15 15:00:14 1999
--- ldap/servers/slapd/config.c.save	Fri Jan 15 14:57:37 1999
***************
*** 355,360 ****
--- 355,375 ----
  				be->be_updatedn = ch_strdup( cargv[1] );
  				(void) dn_normalize( be->be_updatedn );
  			}
+ 		} else if ( strcasecmp( cargv[0], "updatenorepdn" ) == 0 ) {
+ 			if ( cargc < 2 ) {
+ 				Debug( LDAP_DEBUG_ANY,
+ 		    "%s: line %d: missing dn in \"updatenorepdn <dn>\"
line\n",
+ 				    fname, lineno, 0 );
+ 				exit( 1 );
+ 			}
+ 			if ( be == NULL ) {
+ 				Debug( LDAP_DEBUG_ANY,
+ "%s: line %d: updatenorepdn line must appear inside a database definition
(ignored)\n",
+ 				    fname, lineno, 0 );
+ 			} else {
+ 				be->be_updatenorepdn = ch_strdup( cargv[1]
);
+ 				(void) dn_normalize( be->be_updatenorepdn );
+ 			}
  
  		/* replication log file to which changes are appended */
  		} else if ( strcasecmp( cargv[0], "replogfile" ) == 0 ) {
*** ldap/servers/slapd/delete.c	Fri Jan 15 15:00:14 1999
--- ldap/servers/slapd/delete.c.save	Fri Jan 15 14:57:37 1999
***************
*** 72,81 ****
  	 */
  	if ( be->be_delete != NULL ) {
  		/* do the update here */
! 		if ( be->be_updatedn == NULL || strcasecmp( be->be_updatedn,
! 		    op->o_dn ) == 0 ) {
  			if ( (*be->be_delete)( be, conn, op, dn ) == 0 ) {
! 				replog( be, LDAP_REQ_DELETE, odn, NULL, 0 );
  			}
  		} else {
  			send_ldap_result( conn, op, LDAP_PARTIAL_RESULTS,
NULL,
--- 72,85 ----
  	 */
  	if ( be->be_delete != NULL ) {
  		/* do the update here */
! 		if ( be->be_updatedn == NULL || 
! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
! 		     (be->be_updatenorepdn && strcasecmp(
be->be_updatenorepdn, op->o_dn ) == 0)) {
  			if ( (*be->be_delete)( be, conn, op, dn ) == 0 ) {
! 			  	if (!be->be_updatenorepdn || 
! 				    strcasecmp( be->be_updatenorepdn,
op->o_dn ) != 0) {
! 				  	replog( be, LDAP_REQ_DELETE, odn,
NULL, 0 );
! 				}
  			}
  		} else {
  			send_ldap_result( conn, op, LDAP_PARTIAL_RESULTS,
NULL,
*** ldap/servers/slapd/modify.c	Fri Jan 15 15:00:14 1999
--- ldap/servers/slapd/modify.c.save	Fri Jan 15 14:57:37 1999
***************
*** 156,162 ****
  	if ( be->be_modify != NULL ) {
  		/* do the update here */
  		if ( be->be_updatedn == NULL ||
! 			strcasecmp( be->be_updatedn, op->o_dn ) == 0 ) {
  
  			if ( (be->be_lastmod == ON || ( be->be_lastmod ==
UNDEFINED &&
  				global_lastmod == ON ) ) && be->be_updatedn
== NULL ) {
--- 156,164 ----
  	if ( be->be_modify != NULL ) {
  		/* do the update here */
  		if ( be->be_updatedn == NULL ||
! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
! 		     (be->be_updatenorepdn && 
! 		      strcasecmp( be->be_updatenorepdn, op->o_dn ) == 0)) {
  
  			if ( (be->be_lastmod == ON || ( be->be_lastmod ==
UNDEFINED &&
  				global_lastmod == ON ) ) && be->be_updatedn
== NULL ) {
***************
*** 163,171 ****
  				add_lastmods( op, &mods );
  			}
  			if ( (*be->be_modify)( be, conn, op, odn, mods ) ==
0 ) {
! 				replog( be, LDAP_REQ_MODIFY, dn, mods, 0 );
  			}
! 
  		/* send a referral */
  		} else {
  			send_ldap_result( conn, op, LDAP_PARTIAL_RESULTS,
NULL,
--- 165,176 ----
  				add_lastmods( op, &mods );
  			}
  			if ( (*be->be_modify)( be, conn, op, odn, mods ) ==
0 ) {
! 			  	if (!be->be_updatenorepdn ||
! 				    strcasecmp( be->be_updatenorepdn,
op->o_dn ) != 0) {
! 				  	replog( be, LDAP_REQ_MODIFY, dn,
mods, 0 );
! 				}
  			}
! 			
  		/* send a referral */
  		} else {
  			send_ldap_result( conn, op, LDAP_PARTIAL_RESULTS,
NULL,
*** ldap/servers/slapd/modrdn.c	Fri Jan 15 15:00:14 1999
--- ldap/servers/slapd/modrdn.c.save	Fri Jan 15 14:57:38 1999
***************
*** 79,90 ****
  	 */
  	if ( be->be_modrdn != NULL ) {
  		/* do the update here */
! 		if ( be->be_updatedn == NULL || strcasecmp( be->be_updatedn,
! 		    op->o_dn ) == 0 ) {
! 			if ( (*be->be_modrdn)( be, conn, op, dn, newrdn,
! 			    deloldrdn ) == 0 ) {
! 				replog( be, LDAP_REQ_MODRDN, odn, newrdn,
! 				    deloldrdn );
  			}
  		} else {
  			send_ldap_result( conn, op, LDAP_PARTIAL_RESULTS,
NULL,
--- 79,93 ----
  	 */
  	if ( be->be_modrdn != NULL ) {
  		/* do the update here */
! 	  	if ( be->be_updatedn == NULL || 
! 		     strcasecmp( be->be_updatedn, op->o_dn ) == 0 ||
! 		     (be->be_updatenorepdn && 
! 		      strcasecmp( be->be_updatenorepdn, op->o_dn ) == 0 )) {
! 		  	if ( (*be->be_modrdn)( be, conn, op, dn, newrdn,
deloldrdn ) == 0 ) {
! 			  	if (!be->be_updatenorepdn ||
! 				    strcasecmp( be->be_updatenorepdn,
op->o_dn ) != 0) {
! 				  	replog( be, LDAP_REQ_MODRDN, odn,
newrdn, deloldrdn );
! 				}
  			}
  		} else {
  			send_ldap_result( conn, op, LDAP_PARTIAL_RESULTS,
NULL,
*** ldap/servers/slapd/slap.h	Fri Jan 15 15:00:14 1999
--- ldap/servers/slapd/slap.h.save	Fri Jan 15 14:57:38 1999
***************
*** 206,211 ****
--- 206,212 ----
  	char	**be_replica;	/* replicas of this backend (in master)
*/
  	char	*be_replogfile;	/* replication log file (in master)
*/
  	char	*be_updatedn;	/* allowed to make changes (in replicas)
*/
+ 	char	*be_updatenorepdn;	/* allowed to make changes no
replication */
  	int	be_lastmod;	/* keep track of lastmodified{by,time}
*/
  	char	*be_type;	/* type of database
*/
Prev by Date: Re: permissions on slapd.conf
Next by Date: RE: A couple of slapd/slurpd replication suggestions.
Index(es):
- Chronological
- Thread