Re: Relative merits of slapd backends

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 02:59 PM 4/7/99 -0500, Paul Makepeace wrote:
>I'm curious whether anyone's had experience with using different hashing
>backends (i.e. gdbm, Berkeley DB various versions, new mdbm) and can provide
>comparisons/anecdotes in terms of memory usage, ability to handle concurrent
>access, indexing speed, locking caveats, reliability etc.

I generally recommend using BerkeleyDB from Sleepycat for
the following reasons:
	1) actively developed and maintained
	2) good reliability
	3) good set of tools
	4) good overall performance

BerkeleyDB also offers a number of advanced features we hope to
take advantage of.  See discusions on -devel concerning back-bdb2.

>Reliability is obviously important and has been a problem for Schlumberger
>(and now Omnes/me ;-). We have OpenLDAP v1.0.2 on Solaris 2.5.1 with DB 1.85
>deployed over several machines around the world serving a directory of about
>60,000 entries. Corrupted indices seem to be a fact of life and typically
>happen during data cleaning which involves a script running over a large
>chunk of the directory and performing many updates. Is this most likely a
>flaky back-end issue or slapd?

I don't recommend using BerkeleyDB v1.  I had been recommending
BerkeleyDB v2.3.16, but v2.7.3 may now be a better choice. Be sure
to compile BerkeleyDB with a compatible set of compiler options.
Pay particular attention to thread options.

>Reproducing indices is really very slow with the above setup: it can take
>several hours on a Sparc Ultra3000 to create 30 or so indexes (each
>typically eq,sub,pres)[1].

Some older versions of ldif2ldbm did write syncs.  This is very
expensive.  I generally build databases on ram disks... (then
move them to a hard disk before running slapd).

>Would a different backend help (besides better
>I/O and more RAM)? Several hours becomes expensive when an index becomes
>corrupt.

Upgrading could help.  A large number of bugs have been fixed since
1.0.2.  Of course, a few new ones have been introduced. :-)  I
recommend you do adequate pre-deployment testing...

>The hard-nosed question at the moment for me is: is there a significant
>benefit/evil consequence going with Berkeley DB v. GDBM (or some other
>contender)?

As BerkeleyDB v1.85 and GDBM are not actively maintained, I
do not recommend them.  I don't know enough about MDBM
to comment on it.

	Kurt