Re: entryUUID

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 03:04 PM 6/16/2005, Pierangelo Masarati wrote:
>Kurt D. Zeilenga wrote:
>
>>Hmmm....  not sure exactly what your trying to do.
>>
>>I'd like slapd(8) to be slave off of anything.
>That's exactly what I was concerned about.
>
>> If LDAPsync is not
>>available on the provider, I'd like syncrepl to use whatever is,
>>even if that means using basic LDAP operations.
>> 
>Exactly.  But I don't want to touch syncrepl (I mean --- the consumer code).
>I'd rather have a layer that massages consumer's requests and (unaweare of being)
>producer's responses to match OpenLDAP's consumer's needs.

To be efficient (as one can be with basic LDAP operations),
I think it likely that you'd have to add a second
consumer mechanism to syncrepl, one that does two searches
for each refresh:
        1) all obtain entries with just UUID/CSN (or just
        timestamps), use this to delete any shadowed object
        which now doesn't exist.
        2) obtain all entries which were modified since
        the last refresh with desired attributes.

Of course, one could just down all the objects every time
(using LDAPsync or not)... but that's a lot less efficient.


>>When syncrepl uses say basic LDAP operations, saying a timestamp
>>based approach, then one can create entryUUIDs and entryCSNs
>>as needed.  The former can be arbitrary (if this slave maintains
>>a copy, or create_from_name otherwise (maybe with DN+createTimestamp)).
>>The latter can be produced from the timestamp (which, depending
>>how one designs the mechanism, should be enough).
>> 
>OK, we are along the same line here as well.  However, I see, for example, that SunOne
>appears to generate a nsUniqueId that looks like 3ec92655-bc6c11d9-8058fbf5-84bfa6ab
>so I guess this is not something OpenLDAP's consumer can handle as is, and I don't want
>to make the code hairy when not required;

I can see rewriting the attribute name (that looks like a UUID to me).
But I cannot see not adding a new consumer mechanism.

>I'd rather move this type of mucking into a
>separate, configurable layer that turns any type of unique identifier into our entryUUID.

Some sort of rewrite layer that sat upon the consumer mechanism
in syncrepl, but I think you still need a second consumer
mechanism for efficiency.


>>Now, if your trying to chain an LDAPsync operation to a server
>>that doesn't support LDAPsync, I think the operation should
>>fail.
>> 
>It does already!  But if I mask the remote server by means of a layer that makes
>it act as a (refreshOnly) sync provider...

see above.


>>But then maybe I got confused by your use of DSA here... 
>:)
>
>>(especially given your mention of back-sql below).
>> 
>That was an example of how I made a dumb database act as a provider
>by generating entryUUID from RDBMS indices and entryCSN... from nothing!
>
>>>the first of all the control itself, but also the structuralObjectClass (which I plan to compute out of the objectClass chain),
>>>   
>>
>>syncrepl shouldn't require structuralObjectClass (or even
>>objectClass).  That is, the consumer should work in face of
>>fractional entries.
>> 
>So why is the consumer explicitly requesting structuralObjectClass in the attribute list?

I consider that a bug.  It was done for checking purposes for
object class violations, but as the system should support
fractional replication should not only ignore such violations
(the master is responsible for such checks) but the shadow
should not even bother with the check.

>>>the entryCSN (which I plan to estimate from a timestamp whatever, or just to set to something higher than the current CSN, at worst, like it's currently done in back-sql), but I need to be able to generate repeatable entryUUIDs (the "reasonably unique" is essentially intended within a context, and it's an issue I'll need to think about; unless I can do something "absolutely unique" and two-way like in back-sql using tables and rows ids, I need a general way to use the DN, at least one-way; the other can be done somehow programmatically).
>>>   
>>
>>Okay, but not sure if there is sufficient overlap here to put
>>this in an overlay.  Seems that we'll have two kinds of backends.
>>Ones which expect the frontend to maintain these attributes
>>and those which, for whatever reason, what to maintain these
>>themself.  Seems we need a backend flag to advise the frontend
>>what to do.  If 0, the frontend would generate them.  If 1,
>>the frontend would assume the backend does.
>> 
>Not sure I get this point.  My intention is to leave the consumer code exactly as it is now;

My point here is that we have a number of backends where
objects can be added via other paths, like via an SQL INSERT
issued externally to back-sql.  It would be nice to provide
some facility to augment these entries with UUID/CSNs(or timestamps).
I see this as a different problem than shadowing off of
non-LDAPsync-supporting DSAs.

><consumer: OpenLDAP 2.3>
>
><replication proxy: OpenLDAP 2.3 + "syncproxy" overlay that massages consumer requests and producer responses>
>
><producer: whatever>
>
>p.
>
>
>   SysNet - via Dossi,8 27100 Pavia Tel: +390382573859 Fax: +390382476497