Some openldap fixes... (fwd)

[Marijn Meijles <marijn@bitpit.net>]

Peter was still not subscribed. He is now, he says. Honest ;)


----- Forwarded message from peter <peter@twistor.admin.lion-access.net> -----

Date: Tue, 19 Sep 2000 19:52:25 +0200
From: peter <peter@twistor.admin.lion-access.net>
To: openldap-devel@OpenLDAP.org
Cc: marijn@bitpit.net
Subject: Some openldap fixes...
User-Agent: Mutt/1.2.4i

Hi,...

several answers:

Randy wrote:
> I do take exception to the NEXTID solution you have.  It seems that
> some dangeous kinds of corruption could occur if an index file is
> pointing to a deleted id2entry record and you reuse that ID, that's
> a problem.  I doubt that you check all the index files, since you'd
> have to scan them too.  Haven't looked at your code, but I don't
> see how you can avoid scanning the entire id2entry list of keys
> to make sure you have the highest one.  Isn't that expensive, especially
> for the f*ing huge databases we're discussing here :-).  I have a
> few hundred thousand in mine.

eruhmm,.. the id2entry DB is a B(+)tree and thus has ordered keys,
therefore the last is always the greatest.

however you are right in that if the index files get corrupted
one might get pointers to wrong entries,.. but then again,..
IF they get corrupted they might get wrong anywayz

ps,.. a wild guess from my side is a million per database
one of the larger is 43Gb, however due to the fact that it's a
high-available system it's somewhat difficult to count, being
that to do so would require downtime. It's all email though, thus
if one would devide that 43Gb by the avg email size one has a
reasonable estimate.

Kurt wrote:
>>relational table where:
>>    tablename:  <attribute>
>>    key:        <prefix><value>|reverse_string( <dn>)
>>    data:       id-list of all elements that match:
>>                    (<attribute>=<value>) && (<parent dn>=<dn>)
>>
>>    algorithm:
>>        switch( scope)  {
>>            case onelevel:
>>                return idl from key( <prefix><value>|string_reverse( <base dn>))
>>                break;
>>
>>            case subtree:
>>                return merged idls from
>>                    key( <prefix><value>|string_reversed( <base dn>*))
>>                // NOTE: due to the fact that B(+)trees have ordered keys
>>                // this is a linear key seq. starting on the key greater or
>>                // equal to the onelevel key. The iteration stops when the
>>                // keys don't start with the onelevel key anymore.
>>                break;
>>        }
> How do you find the above key if the base isn't the parent?
>
> That is, I have "uid=a,cn=b,cn=c,o=foo" (suffix of o=foo) and
> I do a search -b "cn=c,o=foo" -s subtree "(uid=a)".
>
> I do a lookup for:
>   <prefix>a|string_reverse(cn=c,o=foo)
> but get zero keys returned as I only generated:
>   <prefix>a|string_reserve(cn=b,cn=c,o=foo)
	
The trick is in _greater or equal_ !

read this excerpt from the Sleepycat docs:

int
DBcursor->c_get(DBC *cursor,
    DBT *key, DBT *data, u_int32_t flags);

<snip>
   flags
<snip>

   DB_SET_RANGE
          The DB_SET_RANGE flag is identical to the DB_SET flag, except 
          that the key is returned as well as the data item, and, in the 
          case of the Btree access method, the returned key/data pair is
          the smallest key greater than or equal to the specified key 
          (as determined by the comparison function), permitting partial
          key matches and range searches.

I hope this is clear enough :)

regards,

Peter Zijlstra

----- End forwarded message -----

-- 
Marijn@bitpit.net
---
Why is it illegal to park in a handicapped parking space but okay to
go the bathroom in a handicapped stall?