[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Getting around the single-threaded syncrepl model?



Bannister, Mark wrote:
--On Friday, November 20, 2015 6:31 PM +0000 Albert Braden <abraden@about.com> wrote:

Hi Quanah,

Are you sure your issues with syncrepl aren't specific to Zimbra? When
I ran the Zimbra at Homestead/Intuit we saw syncrepl issues, but I
have not seen those issues in non-Zimbra LDAP clusters.

Quite certain.  For one thing, I file very few syncrepl related ITSes, since I don't use it, but many are filed... For another,
while I have encountered a very few issues with delta-syncrepl, the majority of issues I find w/ Zimbra and replication
are related to syncrepl when it has to be used in an initial or fallback
scenario.

What sort of issues?

And here's some quick stats:
$ grep syncrepl CHANGES  | grep -v delta | wc -l
76

$ grep delta-syncrepl CHANGES  | wc -l
7

Or syncrepl has had 10x times the issues.

Or syncrepl has 10x the users, more eyes spot more bugs, and is now more stable because it has had more fixes?
I'm just speculating here, but I wouldn't be more confident in product Y because it is mentioned less in the change
log than product X.

However, this is a tangent, and still does not help me with my original question.  Correct me if I'm wrong, but
delta-syncrepl is also going to be running in a serial fashion on the master server, and so, if I'm replicating
out to hundreds of replicas, high network latency or timeouts on a single replica would still have a big impact
on my ability to replicate everywhere within 5 minutes.

Syncrepl/delta-syncrepl doesn't make a difference here.  Unless someone has a better suggestion, as I don't
suppose OpenLDAP is going to be fixed to do this in parallel anytime soon, I will go ahead and launch multiple
replica processes on the master server as I proposed at the beginning of the thread, and then balance the
load across these processes to reduce the impact of any one replica going slow.

Certainly won't be changing in 2.4. Very likely to change in 2.5.

https://www.openldap.org/its/index.cgi/Incoming?id=8227

(Or investigate other directory server products that can replicate in parallel ...)

Have fun investigating.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/