RE: bdb backend - reliable or not?

["Matthew Hardin" <mhardin@symas.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org [mailto:owner-openldap-
> software@OpenLDAP.org] On Behalf Of Tomasz Chmielewski
> Sent: Thursday, July 20, 2006 1:24 AM
> To: openldap-software@OpenLDAP.org
> Subject: bdb backend - reliable or not?
> 
> Previously I was using OpenLDAP 2.1.x and 2.2.x with ldbm backend.
> 
> I never had problems with it.

By design back-ldbm cannot detect errors and inconsistencies in the database
that were introduced by system or slapd crashes. Instead, you are left to
find them by yourself at 3:00 AM after chasing missing entries for 18 hours.
 
> As I've read on the list that ldbm support will be dropped in the next
> major version of OpenLDAP, and bdb is the recommended backend, I decided
> to deploy a couple of new OpenLDAP servers with bdb backend.
> 
> 
> Unfortunately, the choice was a disaster - a couple of tests showed that
> after powering off the server, OpenLDAP is not able to start anymore -
> all I see in logs is a single message (loglevel 256):
> 
> bdb_db_open: unclean shutdown detected; attempting recovery.
> 
> And that's it, it never recovers.
> 
> 
> I'm using a pretty old OpenLDAP version, 2.3.6, so my first guess is
> that the problems are because of it, and I'm on my way to upgrade.

That is a good idea. In version 2.3 back-bdb and back-hdb detect improper
shutdowns (server crash, process crash, kill -9, etc) of the slapd process
and perform an automatic recovery of the database during the next startup.
In early releases of 2.3 there were problems with that code.

With OpenLDAP 2.3 it is NOT a good idea to run db_recover from the OpenLDAP
startup script. The slapd process does this automatically by itself.

If you are using one of the platforms we support, you are welcome to try CDS
Silver- it will save you the headache of building and testing OpenLDAP and
is available at no charge.
 
> On the other hand, I didn't have problems with ldbm on 2.3.6, or on any
> older OpenLDAP version, so can one conclude that ldbm is more reliable
> and crash-resistant?

Per my comments above, ldbm simply does not tell you that there are problems
with your database or that it lost entries after a crash. That is one of the
key reasons it is being removed from OpenLDAP.
 
> Will an upgrade to 2.3.24 solve my problems with bdb, or shall I revert
> back to ldbm?

It is very likely that upgrading to a current release of OpenLDAP will
resolve the particular problem you are describing. You may encounter others
as you learn to properly configure and tune back-bdb, but if you pay careful
attention to the documentation for back-bdb and Berkeley DB you will be
rewarded with fast, stable, and reliable LDAP server.
 
> Performance is not an issue here, databases are relatively small.
> What is important is the ability to survive unexpected system
> crash/poweroff.

If that's the criteria then back-bdb and back-hdb are your best choices. In
your case back-bdb will probably suffice. For best results be sure that your
installation includes an appropriate shutdown script so that slapd is
properly terminated.

Cheers,

Matthew Hardin
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
http://www.symas.com

 
> 
> --
> Tomasz Chmielewski
> http://wpkg.org