Re: (ITS#6545) delta-syncrepl rejects modification master accepted

[Frank.Swasey@uvm.edu]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--0-1015574222-1491391973=:71829
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8BIT

Thanks for the patch to provide a test script that just shows the same 
thing.

I see two possible solutions:

  1) replacing the same attribute twice in the same modify LDIF is 
illegal (as it was in older releases)
  2) the accesslog/syncrepl needs to record the final result, not every 
bit of the change.

I believe the easiest solution is #1.  However, #2 would be proper if 
this kind of abuse is legal.

Consider the following deck:

dn: cn=Oh Noes,ou=People,dc=example,dc=com
changetype: modify
delete: sn
sn: George
-
add: sn
sn: Thomas
-
delete: sn
sn: Thomas
-
add: sn
sn: George
-

Is that legal?  Well, it works and results in an entry in the accesslog 
that contains:

reqMod: sn:- George
reqMod: sn:+ Thomas
reqMod: sn:- Thomas
reqMod: sn:+ George

and has no issues in the replication.

Therefore, why does this deck:

dn: cn=Oh Noes,ou=People,dc=example,dc=com
changetype: modify
replace: sn
sn: Thomas
-
replace: sn
sn: George
-

which creates an accesslog entry that contains:

reqMod: sn:= Thomas
reqMod: sn:= George

Interestingly, this morning when I was performing this, the replica ate 
that accesslog entry and put both of those sn's into the LDAP entry on 
the replica (instead of the normal barking in the logs that it failed). 
Perhaps because I had done the former test with the 
delete/add/delete/add sequence?

At this point, I'm thinking the accesslog should probably generate the 
minimal changes necessary to get to the right state.  That would mean 
the former test should produce:

reqMod: sn:- George
reqMod: sn:+ George

and the latter:

reqMod:sn:= George

Either that, or the syncrepl processing of what accesslog is sending 
needs some serious investigation.  I'll see what I can find for time to 
look into that serious investigation.

  - Frank

On Tue, 4 Apr 2017 at 1:54pm, OndÅ?ej Kuzník wrote:

> The attached patch updates the testsuite to reproduce the issue:
> ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170404-ITS-6545-testcase.patch
>
> The issue is that there is information lost when creating the log entry:
> - the ordering of values is not guaranteed by LDAP, even though this is
>  rarely an issue with OpenLDAP with main backends (unless some overlay
>  interferes)
> - there is no way to record where one modification ends and another
>  begins
>
> The obvious way to fix this is to record that a modification has ended.
> This is technically a change to the accesslog format but delta syncrepl
> will recover by falling back to plain syncrepl (which it already does in
> the reported case).
>
> We could also record the order modifications as well (implementing the
> X-ORDERED trait on the attribute), but that would break most consumers
> of this format.
>
>

-- 
Frank Swasey                    | http://www.uvm.edu/~fcs
Sr Systems Administrator        | Always remember: You are UNIQUE,
University of Vermont           |    just like everyone else.
   "I am not young enough to know everything." - Oscar Wilde (1854-1900)
--0-1015574222-1491391973=:71829--