Re: Corrupt LDAP DB ...

[Jose Ildefonso Camargo Tolosa <ildefonso.camargo@gmail.com>]

Hi!

On 11/5/05, Buchan Milne <bgmilne@gmail.com> wrote:
> On 11/5/05, Jose Ildefonso Camargo Tolosa <ildefonso.camargo@gmail.com> wrote:
> > Hi!
> >
>
> [...]
>
> >
> > I think it is.  OpenLDAP almost depends on Berkley DB, so it should
> > give some info in the "general" documentation (there is info, in the
> > FAQ-o-matic), or at least warn about how fragile BDB is.
>
> Recent versions (2.3.x) warn (to syslog) if you don't have a DB_CONFIG
> file for any bdb or hdb database.

Cool.  It was about time, it took two versions to implement that
(please note that I actually give support for OpenLDAP, so I'm not new
to it).

>
> > If you install PostgreSQL, you will get a robust database with almost
> > no extra configuration, it is not the same with berkley, you need to
> > configure the "DB enviroment", wich is in part done by OpenLDAP the
> > first time it opens the DB enviroment, and the DB_CONFIG is
> > database-specific, there is not a "global DB_CONFIG", you have to
> > create a DB_CONFIG for each database you create, and it will depend on
> > the use of the DB.  I think that OpenLDAP should ship with an example
> > DB_CONFIG, and describe for wich "directory size" is that suited.
>
> Recent versions do.

2.3.x do, 2.2.x doesn't.  I don't think it would take a lot of space
to include a sample DB_CONFIG with OpenLDAP 2.2.  And yes, recent
distro packages use to include it.

>
> > > And the major point here, is that your problems stem from two things:
> > >
> > > 1) Using RedHat's distribution of OpenLDAP which is *known* to have
> > > problems, and that is the fault of the distributor, not OpenLDAP
> >
> > I don't use RedHat, and I don't recoment it's use.
>
> There's nothing wrong with RedHat as a distribution. Just don't use
> their packages (at least RHEL2.1, RHEL3, RHEL4 and any Fedora up to
> FC3 at least).
>
> I use Red Hat, but I use the Mandriva packages (which I maintain)
> rebuilt on Red Hat (RHEL3 and RHEL4, our RHEL2.1 boxen still have
> packages based on the RH 2.0.27 packages from RHEL3 but heavily
> modified). My 2.3.11 packages for RHEL are available ...

Yep, but that makes you loose the "warranty" over the packages.  As I
said: I don't use RedHat since version 8.0 (RedHat 8.0) and I tried
Fedora Core 1 and 2, then I decided no to waste my time using them. 
RHEL gives a "warranty" over the binaries, but only if you use the
"repository" packages, when you install a 3th party package, you loose
the warranty over the packages you replace and the packages wich
depends on them.

>
> > I'm using Debian
> > and Gentoo.  From these, only Debian (sarge) have a stable
> > OpenLDAP2.2/BDB4.2
>
> Read on ...

huh? Ok, reading.

>
> > > and
> > >
> > > 2) A lack of basic sys admin skills on your part
> >
> > BDB administration is not that basic.  But I agree with you, one need
> > to read a lot before implementing something, but, by the time I
> > started using OpenLDAP, there was a lack of documentation on what
> > respect to the "fragility" of BDB, and the need to config a DB
> > enviroment (come on, you *need* to configure the DB enviroment, and
> > that's not in the quick start guide, nor in the admin guide).
> >
> > >
> > > Now you can attribute that to 'lack of documentation' and other stuff all
> > > you want.  The fact is, your problems do not stem from the OpenLDAP
> > > project, or a lack of documentation.  And yes, you emailed the list to get
> > > help, which was provided to you, and which for quite some time you decided
> > > to ignore.  And at this point, I'm through with this discussion.
> >
> > I agree on this, I have got help on this list, and I have listen to
> > them, and implemented their advices (wich have solved most of my
> > problems) but I think that one should be worring about other things
> > than the "directory DB"....... I would like to be able to configure
> > OpenLDAP with BDB backend to be as stable as PostgreSQL.
>
> I think your issue may be something else ...

At the moment I dont' have an "issue" here, it is working fine with
debian, but you have to "tune" the DB_CONFIG for large directories.

Berkley DB 4.2.x with openldap 2.1.x and 2.2.x have had a lot of
problems, and you can't deny that.  Today, you can have a very stable
LDAP deployment, but it still have some issues (as of OpenLDAP 2.2.x),
I'm starting tests with openldap 2.3.x.

>
> >  I have
> > configured PostgreSQL databases wich have run without human
> > intervention for over two years, but with berkley
>
> I assume that by "berkley" you actually mean OpenLDAP on bdb ...

Nope, everything that uses berkley requrires a DB enviroment
configuration, whether you configure it with "DB_CONFIG", or you "hard
code" it, you *have* to configure your DB enviroment.

>
> > , I have had
> > databases that crash without being updated, just power up, and
> > shutdown (cleanly), I have a quite stable Debian system now, but It
> > took some time to get there, my gentoo box keeps trashing the BDB
> > (with the same DB_CONFIG and data).
>
> This sounds very much like your slapd is being stopped badly, and not
> having database recovery run for its databases.

Please, read correctly: clean shutdowns (exit with signal 15, not 9,
nor power fail, nor system crash).  I have had this problem even with
a "no shutdown" database, just sit there, an uptime of 23 days, and
then you have some database problems, I have experienced this
(recently) in gentoo, at the moment debian sarge is just fine.  This
is most likely caused by "locks", or something like that (the thing is
that, when you take a look at the DB locks, you are not out of locks!,
but then you shutdown openldap and run db_recovery, and everything
works fine again).

>
> Note that 2.3.x recovers databases at startup, but for 2.1.x and 2.2.x
> manual db_recover is needed if it is likely that slapd could not
> cleanly close all its databases. Debian's init script does this (I
> think you may have to configure it to do so in /etc/default/slapd),
> Mandriva's init script does this by default for 2.2.x (can't remember
> for old 2.1.x packages).

Once again, read completly, I have already said that Debian Sarge's
OpenLDAP works just fine.  I was playing around with gentoo, and just
saw that problem, and i'm trying to isolate it (just to contribute
with an usefull bug report).

Ok, I think this went off-topic enough.

I like OpenLDAP, that's why I'm using it, but Berkley DB have been an
issue since the first time I used OpenLDAP, and it was out of the docs
for a very long time.  I'm glad that it is there now, and that
OpenLDAP is becoming a more stable directory every day.

c-ya!

Ildefonso.