replacement backend (Was: Some openldap fixes...)

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 11:01 PM 9/18/00 +0200, Marijn Meijles wrote:
>You wrote:
>> 
>> Knobs are nice... the new backend will be highly configurable...
>> (if only I had more time to work on it).
>> 
>Maybe you could share some code/ideas with the list, so we can comment and think
>with you.

I'll be committing development incrementally... I do
post request for comments occasionally and am generally
responsive to public enquiries.

The primary design goal of this effort is provide a reliable,
robust directory backend capable of holding million plus
entries per database.   The backend will leverage back-ldbm,
but be designed to take full advantage of the services offered
by the underlying database.

I have chosen to use Berkeley DB 3.1 as the underlying
database manager for the replacement backend.  Berkeley
DB 3.1 offers the necessary transaction capabilities and
is capable of holding 4M key/data pairs per database
(using multiple underlying databases, we should be able
to support 4M entries).

I am currently "piecing together" a prototype.  That is,
we have most everything we need implemented in back-ldbm
and back-bdb2 (an earlier prototype).  As I piece together
things, I intend to rewrite many of the pieces to gain
better reliability and performance.  As the new code
will use DB transaction, the new code will take advantage
of rollback capabilities.

Primary targets of major rewrites will be entry cache
and index management.

A lot of other changes will be made to improve reliability
and performance.  For example, entries will be DER encoded
instead of LDIF encoded to reduce parsing time.

Kurt