Re: (ITS#6545) delta-syncrepl rejects modification master accepted

[ondra@mistotebe.net]

On Wed, Apr 05, 2017 at 07:32:46AM -0400, Frank Swasey wrote:
> Thanks for the patch to provide a test script that just shows the same
> thing.
> 
> I see two possible solutions:
> 
>  1) replacing the same attribute twice in the same modify LDIF is illegal
> (as it was in older releases)

AFAIK, LDAP doesn't forbid it so I don't see that going away.

>  2) the accesslog/syncrepl needs to record the final result, not every bit
> of the change.

Two problems:
- the log is meant to record the request for review/statistics purposes
  as well and should record the intent, not just the result
- the modification might fail, in that case you still need an accurate
  record of the request

> I believe the easiest solution is #1.  However, #2 would be proper if this
> kind of abuse is legal.
> 
> Consider the following deck:
> 
> [...]
> 
> Is that legal?  Well, it works and results in an entry in the accesslog that
> contains:
> 
> [...]
> 
> and has no issues in the replication.
> 
> Therefore, why does this deck:
> 
> dn: cn=Oh Noes,ou=People,dc=example,dc=com
> changetype: modify
> replace: sn
> sn: Thomas
> -
> replace: sn
> sn: George
> -
> 
> which creates an accesslog entry that contains:
> 
> reqMod: sn:= Thomas
> reqMod: sn:= George
> 
> Interestingly, this morning when I was performing this, the replica ate that
> accesslog entry and put both of those sn's into the LDAP entry on the
> replica (instead of the normal barking in the logs that it failed). Perhaps
> because I had done the former test with the delete/add/delete/add sequence?

Your original example did two replacements with the same value, the test
case runs two replacements with different values as that will cause the
two servers to de-sync silently.

> At this point, I'm thinking the accesslog should probably generate the
> minimal changes necessary to get to the right state.  That would mean the
> former test should produce:
> 
> reqMod: sn:- George
> reqMod: sn:+ George
> 
> and the latter:
> 
> reqMod:sn:= George
> 
> Either that, or the syncrepl processing of what accesslog is sending needs
> some serious investigation.  I'll see what I can find for time to look into
> that serious investigation.

It does need a fair bit of attention, the problem is it's something that
would require a format change and there are already existing consumers
that would be affected.

I'm going to try with a patch to record something like this:
regMod: sn:= George
regMod: :
regMod: sn:= George

Together with the relevant syncrepl updates.

While consumers would still be affected, this should be quite rare and
the current handling of that case would have been incorrect anyway. The
change would then only make the silent failure a noisy one.

-- 
OndÅ?ej Kuzník
Senior Software Engineer
Symas Corporation                       http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP