Re: My database are often corrupted.

[Michael Anderson <mikea@tuatara.bm.uunet.de>]

On Thu, 2004-08-12 at 21:50, Arnault Teissier wrote:
> On jeu, 2004-08-12 at 10:55 +0200, Buchan Milne wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > Arnault Teissier wrote:
> > | Hi,
> > |
> > | i really worry about our ldap master server... The databases are
> > | corrupted in average, one time a month. And never the same, is change
> > | regulary. Generally, the server don't respond anymore, or sometimes, it
> > | continue to run, but initiate many thread, and use 100% of CPU. I check
> > | our configuration, but i think that it's ok.
> > |
> > | We're running 3 databases, and on one server. Today again, a db crashed.
> > | So i decide to contact you, because it can't continue like that. Ldap is
> > | the center of our architecture, and without it, nothing work...
> > | I try to recover the database with db_recover, but nothing change...
> > | It's always the same. If i launch slapd, it start, and freeze. So i need
> > | to kill -9 the process. it doesn't open the 389 port.
> > | I join you the result of a db_recover, and the log generated when i
> > | launch slapd -d -1can be found at
> > http://www.assonetworx.com/slapd.debug.log
> > |
> > | After many research, i wasn't able to find a solution in my problème.
> > | I'm using the sid debian package of openldap.
> > | slapd 2.1.30, and BDB 4.2.52.
> > |
> > | regards
> > |
> > | --- db_recover outputs ----
> > |
> > | root@ulysse:/var/lib/ldap/assonetworx $db_recover -v
> > | db_recover: Ignoring log file: log.0000000001: unsupported log version 8
> > | db_recover: log_get: unable to find checkpoint record: no checkpoint
> > | set.
> > | db_recover: Ignoring log file: log.0000000001: unsupported log version 8
> > | db_recover: Recovery complete at Mon Dec 23 10:00:20 1935
> > | db_recover: Maximum transaction id 80000000 Recovery checkpoint [0][0]
> > |
> > | -------
> > |
> > 
> > Please add a checkpoint setting to the database section of your
> > slapd.conf for this bdb database.
> 
> I put my slapd.conf minus some sensible information on :
> http://www.assonetworx.com/slapd.conf
> 
> (If you think i should remove some other stuff, just tell me ;) )
> 
> > (I suspect what is happening is that you are reaching 2GB of active
> > transaction log files, which seems to cause problems ...).
> 
> I don't think so, because my transaction log files is smaller than that.
> About 20-30 Mb.
> 
> > Also, it seems you have some transaction log files created when using an
> > earlier version of Berkeley DB, you should at least run db_checkpoint
> > before chanding Berkeley DB versions (which should flush the
> > transactions to the database so you can remove the transaction log file).
> 
> It could be the problem, because i'm using Debian/Sid package, and maybe
> the maintener didn't noticed it. I quite often upgrade my slapd, and
> Bercley DB...
> But i've a question, if my log file is corrupted, all my database is
> corrupted? because if I remove this file, it doesn't change anything, my
> corrupted database, is still dead.
> 
> Regards

We had similar problems and as a temporary measure returned to ldbm,
Interestingly, ldbm seems to use some of the same libraries but since we
reverted we have not had a single problem. If you do not really need the
transaction capability of Berkeley this may be an option for you.

In the end we decided that openLDAP is not really suitable for
mission-critical apps and have since made the decision to go to Novell
e-directory. You might like to look at that because although it is
commercial, it is not terribly expensive and blows the socks off
openLDAP.

Regards,

Mike Anderson