Re: syncrepl ramblings again

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 06:42 PM 8/28/2004, Jong-Hyuk wrote:

>> >Assuming that entryCSN and entryUUID are supported by an LDAP server,
>> >a possible approach to generic slapd sync consumer is:
>> >
>> >(1) Search for (&(original filter)(entryCSN<clientCookie)) asking only
>> >entryUUID and entryCSN attributes
>> >(2) Delete replica entries not returned by (1)
>> >(3) Search for (&(original
>> >filter)(entryCSN>clientCookie)(entryCSN=<maxCSN(1))
>> >      asking replicated attributes + entryUUID + entryCSN,
>> >      where maxCSN(1) is the maximum entryCSN returned in step (1)
>> >(4) Update clientCookie to maxCSN(1)
>> >
>> >As a result, replica is synchronized to the point maxCSN(1)
>> >
>> >This in fact emulates the present mode LDAPsync operation without
>LDAPsync
>> >support in the provider side.
>> >
>> Sounds good to me, except step 3 with maxCSN makes no sense to me.
>
>(entryCSN<=maxCSN(1)) is required to ensure proper ordering observed at the
>consumer. It will make the consumer be consistent to a provider state at
>point (1).

This cannot be guaranteed, in general, as a search operations
don't operate in isolation to modifications.  That is, a search
can return an entry with CSN+1 but not the entry not CSN+0 even
though the latter was committed first.  However, you can
guarantee eventual convergence.

>Subtractive changes are always problematic without tombstone /
>changelog. If an entry is deleted between (1) and (3), the consumer state
>will not be consistent to any one of the provider states which have existed.
>Because consumer is only aware of deletions up to the point (1), it also
>should be aware of additions and modifications only up to that point, not a
>single one more.
>
>> Step 1 must of course be
>>    (&(original filter)(entryCSN<=clientCookie))
>>
>> and step 3 we'd like to use '>' but LDAP only allows '>=' so
>>    (&(original filter)(entryCSN>=clientCookie))
>
>let me be more exact.
>
>(1) Search for (&(original filter)(entryCSN<=clientCookie)) asking only
>entryUUID and entryCSN attributes
>(2) Delete replica entries not returned by (1)
>(3) Search for (&(original
>filter)(!(entryCSN<=clientCookie))(entryCSN<=maxCSN(1))
>      asking replicated attributes + entryUUID + entryCSN,
>      where maxCSN(1) is the maximum entryCSN returned in step (1)
>(4) Update clientCookie to maxCSN(1)
>
>> As for syncrepl itself: from seeing all the traffic during the startup
>> of a consumer, it would be nice to avoid step 1. What was the motivation
>> for not keeping tombstones around? It seems a small price to pay,
>> compared to the traffic overhead every time a consumer starts up.