[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Syncrepl: full sync vs. delta
I recently had to process some 40,000+ modifications through my set of
directory servers. Initially, I tested my changes through my dev boxes
(2.3/HEAD using syncrepl). This run took approximately 2 hours, but the
slaves were modified essentially as the master was. This length of time
concerned me somewhat going forward, but on my production boxes (slurpd
master->slave) it took only 45 minutes.
I discussed this time disparity with Howard, and he made some modifications
to syncrepl (full sync mode) that allowed the master to take the changes in
around 16 minutes. However, the slaves still took several hours to catch
up on these modifications, which meant they were out of sync for long
periods of time. Howard and I then discussed putting together the syncrepl
delta method, using the accesslog backend (as previously discussed on
-devel). This worked (after some bugs in accesslog were fixed), where it
took 37 minutes to push the updates through the master. 2 of the 3
replica's finished within a few seconds of the master, and the 3rd slave
finished within 5 minutes of the master.
However, the problem with the way we currently have to set up
delta-syncrepl (via accesslog) is that there is no way for a slave to
become fully refreshed if its contextCSN is out of date. It looks like
this would take an extension to the syncrepl protocol for this to be done
properly. Objections? comments?
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
"These censorship operations against schools and libraries are stronger
than ever in the present religio-political climate. They often focus on
fantasy and sf books, which foster that deadly enemy to bigotry and blind
faith, the imagination." -- Ursula K. Le Guin