Re: Need new replica configuration without shutting down slapd

[Ganesan R <rganesan-ldap@myrealbox.com>]

>>>>> "Howard" == Howard Chu <hyc@highlandsun.com> writes:

>> -----Original Message-----
>> From: Ganesan R <rganesan-ldap@myrealbox.com>
>> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Ganesan R

>> Hi,
>> 
>> I have a need for adding/removing OpenLDAP replicas in a
>> multi-master setup
>> with two requirments

>> 1. Minimizing (or totally avoiding) config file changes.
>> 2. Automatic new replica creation especially without restarting any slapd
>> instance.
>> 
>> To accompish 1, I am working on a design which will put replica
>> information
>> inside the database. This way replica information propagates to
>> all servers
>> by normal replication instead of by modifying individual conf files. I am
>> making reasonable progress with this.

> Sounds interesting. Kind of like my slapd.conf backend. I haven't come up
> with sensible semantics for all the possible operations on it, perhaps I
> set my initial sights too high...

What I have done so far is to put the equivalent of the replica line in
slapd.conf into a "replica" object (per host). Each suffix that needs to be
replicated is associated with an aux class "replicaContext" and a list of
DNs pointing to the replica objects. 

So far I've managed to get slapd successfully get read this information to
populate it's internal replica data structures at startup. I've also
modified slapd to handle writes to the above "replica DN" and reconfigure
it's internal data structures. So, replica addition/deletion can now be done
by writing to a special attribute. I am wondering if it's better to
implement a control to add/remove replicas. 

I am looking at slurpd now. I am wondering why slapd bothers with the
replica information at all. All it needs to know is whether a replica log
entry needs to be written. Instead of slapd writing "replica:" lines to the
log, we can move all this intelligence to slurpd itself. There's no problem
in restarting slurpd, may be by just sending it a signal to reconfigure it's
replicas. 

Btw, your slapd.conf backend sounds quite interesting. Do you have a draft
design and/or code that I can look at?

>> I now need a way of achieving 2. My idea is to bring up the new replica
>> server with a minimal database and have an application on the other side
>> (probably by a modified slurpd) to read the database (similar to
>> slapcat) and
>> bring the replica uptodate.
>> 
>> Is it okay for a tool to read the database in read only mode while the
>> slapd is still running? Will there be a need to flush the
>> database cache in
>> slapd or is it okay to configure slapd to flush after every update?

> The default behavior for back-ldbm is to flush after every update, unless
> you use the "dbnosync" keyword in the configuration. However, depending on
> what DBM library you use, it may not be possible to access the database
> while slapd is running. For example, if back-ldbm is using gdbm, the
> database will be opened with a write-lock. If you use gdbm in another
> application, it should be unable to obtain a read-lock on the database. In
> general, with back-ldbm, only the slapd process should have access to the
> database.

What alternative is there to get at all the attributes (including
operational attributes like creationTimeStamp) of a database? I need a way
to essentially create a replication log for the entire database. May be
slurpd can do a "normal" ldap search on the localhost with a special
control?

Ganesan