Re: syncrepl ramblings again

["Jong-Hyuk" <jongchoi@OpenLDAP.org>]

> >Assuming that entryCSN and entryUUID are supported by an LDAP server,
> >a possible approach to generic slapd sync consumer is:
> >
> >(1) Search for (&(original filter)(entryCSN<clientCookie)) asking only
> >entryUUID and entryCSN attributes
> >(2) Delete replica entries not returned by (1)
> >(3) Search for (&(original
> >filter)(entryCSN>clientCookie)(entryCSN=<maxCSN(1))
> >      asking replicated attributes + entryUUID + entryCSN,
> >      where maxCSN(1) is the maximum entryCSN returned in step (1)
> >(4) Update clientCookie to maxCSN(1)
> >
> >As a result, replica is synchronized to the point maxCSN(1)
> >
> >This in fact emulates the present mode LDAPsync operation without
LDAPsync
> >support in the provider side.
> >
> Sounds good to me, except step 3 with maxCSN makes no sense to me.

(entryCSN<=maxCSN(1)) is required to ensure proper ordering observed at the
consumer. It will make the consumer be consistent to a provider state at
point (1). Subtractive changes are always problematic without tombstone /
changelog. If an entry is deleted between (1) and (3), the consumer state
will not be consistent to any one of the provider states which have existed.
Because consumer is only aware of deletions up to the point (1), it also
should be aware of additions and modifications only up to that point, not a
single one more.

> Step 1 must of course be
>    (&(original filter)(entryCSN<=clientCookie))
>
> and step 3 we'd like to use '>' but LDAP only allows '>=' so
>    (&(original filter)(entryCSN>=clientCookie))

let me be more exact.

(1) Search for (&(original filter)(entryCSN<=clientCookie)) asking only
entryUUID and entryCSN attributes
(2) Delete replica entries not returned by (1)
(3) Search for (&(original
filter)(!(entryCSN<=clientCookie))(entryCSN<=maxCSN(1))
      asking replicated attributes + entryUUID + entryCSN,
      where maxCSN(1) is the maximum entryCSN returned in step (1)
(4) Update clientCookie to maxCSN(1)

> As for syncrepl itself: from seeing all the traffic during the startup
> of a consumer, it would be nice to avoid step 1. What was the motivation
> for not keeping tombstones around? It seems a small price to pay,
> compared to the traffic overhead every time a consumer starts up.