Re: "LDAP ease modify restrictions" support

[Radovan Semancik <radovan.semancik@evolveum.com>]

On 02/22/2016 06:43 PM, Michael Ströder wrote:
> > > Whether LDAP result code attributeOrValueExists is treated as an error or just
> > > logged as "nothing to be seen here" is up to your code, isn't it?
> > Depends on a layer. The operation will always be logged as LDAP error response.
> But it's up to your application whether a LDAP response code results in a
> warning/error log level message or a debug/info log level message.

Depends. It is possible to do as you suggest. But it may break code 
layering and encapsulation, ending up in a code that is difficult to 
maintain. But I understand that there are trade-offs to be made ...

> > The connector also does not know whether it is real error or not.
> This sounds strange to me. Connectors I implement for my customers always know
> whether something is a real error or not and then it decides whether to abort or
> ignore.

The ConnId connectors are designed to be simple. There is a good reason 
for that: if most of the functionality is implemented in IDM it does not 
need to be reimplemented in each connector again and again. There is 
just one IDM, but there are many connectors (the connector API is in no 
way LDAP-specific). So the connectors are passing almost all errors to 
the upper layers to be handled by a common processing code.

Yes. I can do it entirely in the connector: handle the error, mute it, 
silently read the entry, recompute the change, modify the operation and 
retry it. But it is an ugly exception, it complicates the client code 
and it actually duplicates the code that is already implemented in the 
IDM. That is not good. Not good at all.

> > Well, that might be enough for web2ldap. But we are a "self-healing" system.
> Self-healing just means that if anything goes seriously wrong in a connector it
> will try to reach the target state by doing a full sync during next run. Nothing
> special in your product.

That's not always feasible. Full re-synch of databases with millions of 
identities just takes too long. Even several hundred thousands can take 
a long time. And the state of the system changes during the resync. When 
one resync ends another may start again :-)  Even a full sync of a 
single identity is not that easy. E.g. there are often attributes that 
are not managed by IDM, some are even source of authoritative data for 
IDM, some attributes even have several authoritative sources, some 
groups are assigned by IDM and some are not, etc. ... So there needs to 
be a continual and quite sophisticated process how to handle 
inconsistencies that are discovered anytime during normal operation. And 
midPoint has several mechanisms to handle that. E.g. if midPoint gets 
"object not found error" it consults the policies, re-creates the 
missing objects and then replays the operation. If midPoint gets 
"already exists" error it reads the entry, consults the policies and 
either links it to existing user or deletes it. Then restarts the 
operation. The handling of attributeOrValueExists error would be a good 
example of this "discovery" mechanism. Except that it is not. Because 
the error stops the operation that would fix the data if it is allowed 
to proceed. So the error just gets into the way without providing any 
benefit.

> > Not even in that case. E.g. see above. You will not get the error if 
> > you are re-adding a group that was deleted just a millisecond ago 
> > just because the network latencies haven't turned up in your favor. 
> Now this sounds suspicious to me. Are you using several MMR replicas to write to
> at the same time? If yes, here's some advice in good faith and without offense:
> DON'T DO THAT!

No, we are not writing to two masters at the same time. The same use 
case also applies to a single-server directory topology.

> > "Hey there! Maybe something wrong happened. Or maybe not. It may all be OK.
> > There is no way to be sure. So forget it. I just wanted to talk to you. Sorry to
> > bother you. And, by the way, your operation failed. Just for fun. Try something
> > else. I won't tell you what. Go figure. Bye."
> There's nothing wrong to log a debug/info message saying:
> "Ignored LDAP response code <foo> because of condition <bar>"

Once again: Ignoring the error is inconvenient, but it is not the 
primary problem. Primary problem is that the operation is stopped and 
unaffected changes are not executed. So it needs to be recomputed and 
retried. In my use case the error is useless and even harmful. It is 
great that there is a control to turn it off. And no, my client code is 
not broken.

--
Radovan Semancik
Software Architect
evolveum.com