[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Syncrepl: full sync vs. delta

At 05:31 PM 10/5/2005, Quanah Gibson-Mount wrote:
>I recently had to process some 40,000+ modifications through my set of directory servers.  Initially, I tested my changes through my dev boxes (2.3/HEAD using syncrepl).  This run took approximately 2 hours, but the slaves were modified essentially as the master was.  This length of time concerned me somewhat going forward, but on my production boxes (slurpd master->slave) it took only 45 minutes.
>
>I discussed this time disparity with Howard, and he made some modifications to syncrepl (full sync mode) that allowed the master to take the changes in around 16 minutes.  However, the slaves still took several hours to catch up on these modifications, which meant they were out of sync for long periods of time.  Howard and I then discussed putting together the syncrepl delta method, using the accesslog backend (as previously discussed on -devel).  This worked (after some bugs in accesslog were fixed), where it took 37 minutes to push the updates through the master.  2 of the 3 replica's finished within a few seconds of the master, and the 3rd slave finished within 5 minutes of the master.
>
>However, the problem with the way we currently have to set up delta-syncrepl (via accesslog) is that there is no way for a slave to become fully refreshed if its contextCSN is out of date.
>It looks like this would take an extension to the syncrepl protocol for this to be done properly.  Objections? comments?

Without specifics as to what part of the LDAP Sync semantics
you wanted to change, I have no comment.  I note that
contextCSNs are not part of the LDAP Sync semantics,
just an implementation detail.  So, I assume you mean the
'cookie' is too old to allow partial update, requirng
a full refresh to be sent.

Kurt


>--Quanah
>
>--
>Quanah Gibson-Mount
>Principal Software Developer
>ITSS/Shared Services
>Stanford University
>GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
>
>"These censorship operations against schools and libraries are stronger
>than ever in the present religio-political climate. They often focus on
>fantasy and sf books, which foster that deadly enemy to bigotry and blind
>faith, the imagination." -- Ursula K. Le Guin