Re: (ITS#5669) slapd replica ignores some new objects

[hyc@symas.com]

ali.pouya@free.fr wrote:
> Full_Name: Ali Pouya
> Version: 2.4.11
> OS: Linux 2.6 (Fedora)
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (145.242.11.3)
>
>
> I have a directory with a master and one replica in RefreshAndPersist mode.
>
> The replica is synchronized with the master. I stop it for making a backup.
> During that time I do three write operations on the master :
>
> I add a new object o1,
> then I modify an already existing object o2,
> and finally I add another new object o3.
>
> After service startup, the replica gets synchronized with the master, and the
> contextCSN attributes are the same on both servers.
> But the object o1 is missing in the replica !
>
> More investigation shows that the sync provider sends objects to the consumer in
> createTimestamp order.
> In other words the sync information is sent in this order : o2, then o1, then
> o3.
>
> After getting o2, the consumer rejects o1 (which has now a smaller entryCSN)
> with this message in the log:
>
> ....
> do_syncrep2: cookie=rid=002,csn=20080822130259.472005Z#000000#001#000000
> do_syncrep2: rid=002 CSN too old, ignoring
> 20080822130259.472005Z#000000#001#000000
> ldap_msgfree
> ...
>
> I think sync data would better be sent in entryCSN order rather than in
> createTimestamp order.

That is not possible, nor is it supposed to be necessary with the way the 
syncrepl protocol was designed. During a refresh (which occurs at server 
startup time, at least) the consumer's context is not updated until all of the 
modified entries have been received. So, this particular comparison should 
never fail like this.

> I precise that I have set the following directives (I have no delete operation
> in my directory and I wish to avoid the present phase to be engaged) :
>
> syncprov-nopresent     TRUE
> syncprov-reloadhint    TRUE
>
> If I comment out at least one of these directives then the problem disapears
> (the object o1 is present in the replica).

You're tripping over a behavior change that was made to fix ITS#5493. For now, 
you should only use both of those settings together if the underlying database 
is an accesslog DB (which always returns entries in modification order). We 
should probably use some other method of detecting the accesslog...

> The environment and configuration files are the same as for ITS 5661.
> Of course I can provide any other information required.


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/