RE: back-bdb DB_RECOVER and soft restart

["Matthew Hardin" <mhardin@symas.com>]

This is a followup to the back-bdb DB_RECOVER thread from last year. We want
to add automatic recovery to back-bdb and propose to solve the problem this
way.

The modifications that follow involve changes to the back-bdb initialization
and shutdown routines. They are intended to detect an improper shutdown of
back-bdb and initiate a recovery only when there are no other instances of
back-bdb accessing the
db. Further, additional instances of back-bdb (i.e., tools) will not
complete their initialization until the the db recovery has been completed.

The mechanism uses a combination of lock files and file locks, and works as
follows:

On startup each instance of back-bdb will do the following:

1. Open the lock file in the db directory called slaplock with O_CREAT.
   This step is only to make sure there is something to lock,
   and it doesn't matter if the file already exists.
2. Attempt to place a write lock on the lock file. If the lock fails,
   it means another back-bdb instance is is either recovering  the db
   or using it, so proceed to step 5.
3. Stat the lock file. If the file size is non zero, it means that
   no other back-bdb instances are using the db and that the db
   was not properly closed, so perform the recovery.
4. Write one byte to the file (one variation is to write the PID into it
   so one can tell by inspection which process did it).
5. Wait for a read lock on the lock file and leave it there for the
   life of the back-bdb instance.
6. Open the db and finish initialization.


On bdb shutdown, each instance of back-bdb will do the following:

1. Attempt to place a write lock on the lock file. If it fails,
   it means that other back-bdb instances are using the db file,
   so go to step 3.
2. Perform the DB shutdown and then truncate the lock file to
   0 bytes. That signals that the db was shut down cleanly.
3. Close the lock file and exit normally.


This appears to cleanly and portably solve the problem of back-bdb
DB_RECOVER and soft restart. In addition to slapd itself, any of the slap
tools that opens a db that was shutdown uncleanly will initiate a
DB_RECOVER, but only if they are the only process accessing the database.
Once recovery is complete, then operation proceeds as normal.

Comments?

Matthew Hardin
Symas Corporation
Packaged, certified, and supported LDAP software:
http://www.symas.net/download

> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Howard Chu
> Sent: Friday, August 09, 2002 4:16 PM
> To: openldap-devel@OpenLDAP.org
> Subject: back-bdb DB_RECOVER and soft restart
>
>
> A couple weeks ago I patched init.c to exclude the DB_RECOVER flag when
> running a slap tool and initializing the BDB environment. This allowed
> slapcat to run concurrently with slapd, and actually I see no reason why
> slapadd or slapindex wouldn't also work since they both operate
> additively.
>
> There's still a problem if you're trying to start a 2nd slapd on
> an existing
> database, ala soft restart. I think we need some kind of a
> semaphore instead,
> such that whenever any program starts, if it's the only one
> accessing the BDB
> environment, it automatically performs a recovery. But if there are two or
> more active instances, the subsequent programs leave it alone. That should
> give us the most safety and convenience, and you can still just run
> db_recover manually if you really need it.
>
> Or we can just ditch the auto-recovery completely and always
> require manual
> use of db_recover instead.
>
>   -- Howard Chu
>   Chief Architect, Symas Corp.       Director, Highland Sun
>   http://www.symas.com               http://highlandsun.com/hyc
>   Symas: Premier OpenSource Development and Support
>
>
>