[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Replication between 2.2.x and 2.3.x

At 01:06 PM 6/15/2006, FRLinux wrote:
On 6/15/06, Jed Donnelley <jed@nersc.gov> wrote: 
What makes you think so?  What testing have you done to suggest that
your 2.3 master won't replicate to your 2.2 slaves?  Slurpd?  Syncrepl?
If Syncrepl, refresh-only?   See our configuration below.


Alright, a bit of background then. Our master was at first a 4.2db
with OpenLDAP 2.2.29 on FreeBSD. At the time we installed our
replicas, they all got updates as soon as we were adding an entry in
the master. We are using syncrepl with refresh-and-persist.

Since the move to the new master, 4.2db with OpenLDAP 2.3.2x, neither
the 2.2 or 2.3 replicas are getting updates. We did add several
entries and they never seem to make it to the replicas. Communication
is no issue, we can connect from/to the master without a hitch.


Interesting.  We are at exactly the same master on the same FreeBSD
OS.  We also have an instance:

Production systems Test systems
2.2.29 Master production ldap ----> 2.3.23 Slave | Master -----> 2.2.30 Slave
| \ /
Existing replicated production slaves (2.2.23, 2.2.24, 2.2.28, and 2.2.29)

, the 2.3.23 Slave | Master - also on FreeBSD, that is successfully
replicating to that 2.2.30 Slave and others.  Though the other testing
is still ongoing, I've been running all our production updates from the
2.2.29 Master through the 2.3.23 Slave | Master out to that 2.2.30 Slave
for about a month now.  Our slaves are running on various FreeBSD,
Linux, and AIX (perhaps others?) systems.  We also used to run on
Solaris, but I don't think we have any Solaris instances left.

The one thing that I see is different is your use of refresh and persist.
Syncrepl with refresh and persist is quite a bit more demanding on
the software as it demands that the server notify the slaves when there
are needed changes.  With refresh only (as we are using) the slaves
are polling for changes.  Since, except for the timing and perhaps
load if you have lots and lots of slaves, the semantics of refresh only
should be like that for refresh and persist (unlike that with slurpd where
you have to initialize the slaves), I suggest you might want to try
refresh-only (change config, restart master, restart slaves).  It seems
to be working fine for us.

We plan to try "upgrading" to refresh and persist once we get our
master to 2.3.x.  At that time we'll try testing a 2.3.x replica with
refresh and persist.  If that works we'll move our slaves to refresh
and persist as they get upgraded to 2.3.x.  At this point I don't
remember why we backed off from our early efforts with refresh
and persist, but we had some problems.  I believe they were more
with crashing and not with replication, but there might have been
some replication problems that I'm not remembering at this point.

So at the moment, we have a master running the latest 2.3 from FreeBSD
ports with db4.2 and replicas varying from 2.2 to 2.3 which do not get
updates and we're slightly puzzled about this since it looks like we
got our setup right.


As far as that goes our setup is the same and we are getting updates
just fine.  We have a watchdog timer that updates every 10 minutes
and many thousands of user records with typically some dozens being
updated daily.  All are going through the whole setup above and we have
monitoring set up on the watchdog timer (beyond manual checking now
and then during our testing).  So far so good for our testing.

I can make available our hacked little watchdog timer scripts (one for writing
and the real hack for reading/checking) if anybody's interested.  Is there a
repository for ldap programming examples somewhere?  Perhaps LDAPMAN.ORG?
We have a bunch of other code (mostly making LDAP consistent with data in other
forms) the we might be able to share with such a repository.

--Jed http://www.nersc.gov/~jed/