Re: Syncrepl: full sync vs. delta

[Howard Chu <hyc@symas.com>]

Quanah Gibson-Mount wrote:
> --On Wednesday, October 12, 2005 9:35 AM -0700 Howard Chu 
> <hyc@symas.com> wrote:
>
> > Howard Chu wrote:
> > > In fact, running test008 with an accesslog overlay that uses a single
> > > mutex on the write ops actually completes faster than normal, because
> > > back-bdb never hits any deadlocks. (Quite a bit faster, 2x at least.)
> > > So I guess this approach is acceptable.
> > The assignment of CSNs to operations needs to occur later though. ModRdn
> > is fine, because it doesn't get a CSN until the backend calls
> > slap_modrdn2mods. But Add, Delete, and Modify all assign the CSN in the
> > frontend, before the accesslog's mutex is locked, so CSNs are still
> > allocated in an uncontrolled order. I'm thinking that all of the
> > assignments should be invoked from the backend. But that means that 
> > other
> > modules that call into backends would need to do the graduate() call 
> > that
> > the frontend does right now. I guess the simplest would be for the
> > accesslog overlay to just replace the CSN the frontend assigned with a
> > new one. Or add a backend flag "SLAP_DBFLAG_DEFER_CSN" that tells the
> > frontend not to assign it, because someone else will. ...?
>
> If someone is going to be using delta-syncrepl with a fallback of a 
> full refresh, don't the CSN's need to be coordinated between the 
> accesslog backend and their main (bdb/hdb) backend?

Yes, that would be best. Currently they are.

>   I'm guessing that is what you mean will happen in the 
> SLAP_DBFLAG_DEFER_CSN bit.

Yes. But that's still too messy; currently only the frontend de-queues 
CSNs but there are several places that en-queue them (passwd.c, sasl.c, 
various overlays) which means right now those places cause a CSN memory 
leak. The cleanest solution now appears to be invoking the CSN 
management directly from the backend (like ModRdn does) and removing the 
responsibility from the frontend. This seems to be the only way to make 
sure slap_graduate_commit_csn() will get called when it's needed.

--
 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/