RE: Bdb defaults - WAS: problem importing entries.

[Michael L Torrie <torriem@chem.byu.edu>]

On Tue, 2004-06-15 at 08:27, Armbrust, Daniel C. wrote:
> I don't mean to keep opening this box of worms, but it just seems like
> the learning curve for setting up an openldap server is unnecessarily
> high.
> 
> When I think back to my first experiences of setting up other
> databases - mySQL, postgres - or even other ldap servers like Tivoli,
> I have been able to simply install them and toss millions of records
> at them, without having to do any performance tuning, reading third
> party manuals, or digging through the full documentation for the
> server itself.  Like openldap, they are all very large complicated
> apps.  But out of the box, they all worked pretty well.  This just
> hasn't been my personal experience with openldap.  

Well said.  While the posts from the OpenLDAP developers do have good
points, your points are also very well taken.  I am no linux newbie, and
also have set up many databases in my day.  I don't know a lot about
tuning, other than tweaking some of the more documented settings and
have always received reasonable performance from those database
servers.  OpenLDAP, however, has alwas caused me problems.  The most
common problem I see is index corruption, or just plain segfaults, both
of which are really hard to reproduce for the developers since they seem
to be triggered by problems in my database (usually the standard
recovery and reindex commands send it on its way again).  We've now
taken to stopping LDAP every night, dumping it out with slapcat (for
backup purposes), running slapindex, and restarting it.  This seems to
prevent some of the indexing problems we've seen, and keep OpenLDAP a
bit more stable.

We are currently in the process of seriously examining Novell's
e-directory because it supports an LDAP personality and can provide
everything (and much more) that OpenLDAP supports, but seems to come
with reasonable defaults that let it work with tens of thousands of
records out of the box, without problems.  Plus the e-directory can be
used without charge for less than 100,000 users, I believe.

I know that the OL developers can't be expected to provide settings for
all, or even many configurations.  But it seems to me that most OpenLDAP
users work with several thousand or less users, so providing example
settings to work with with that kind of load might be appropriate.

I also agree that the docs could be better.  I too have followed the
documentation path that you mention below and have not found it easy to
find everything I need to know to effectively tune the system.

Anyway, I'll keep following the list and trying to pick up the
information I need to make OpenLDAP run.  I think OL is a key product to
the success of open source and free software in the enterprise, although
it seems to be one of the least stable from the perspective of us
linux-savvy admins who are not BDB gurus.

Michael


> 
> Everyone keeps saying that you have to read the manual.  But what do
> you learn, if you read the openldap manual?
> 
> When you look at the documentation right now, as a beginner would,
> they would probably start with the quick start guide.  It gives an
> example using bdb (so at this point, you probably wouldn't even know
> other backends exit) but it says nothing about configuring bdb with a
> DB_CONFIG file.  It doesn't even have a link to some suggested reading
> for improving performance, or even mention that you should create a
> DB_CONFIG file.
> 
> So I go to the admin guide.  There is no apparent section on tuning. 
> One would assume it all happens in the configuration file.  So I go to
> read that section.  It is very good at explaining most things.  But
> when you get the backend section, it simply spits out a laundry list
> of what exists.  No recommendations as to which one to use, no details
> on what each one is good at or even intended for.  Again, no mention
> what-so-ever of DB_CONFIG, or even a link to the appropriate Berkeley
> documentation.  No example configurations.  Even though people have
> said on the mailing list and in the FAQ that openldap doesn't provide
> any defaults for BerkeleyDB, and the BerkeleyDB defaults are
> inappropriate for openldap.
> 
> If I go on to the FAQ before I come to the mailing list there is
> finally a mention of DB_CONFIG, along with some reasonable
> documentation of the options you may want to experiment with.  There
> is also another entry that has some details about each of the
> backends.  But this assumes that I know what I'm looking for in the
> first place, which a new person probably wont know.  And it still
> doesn't have any working copy and paste examples.
> 
> Maybe its just me.  But working example configurations make a
> complicated system _much_ easier to learn.  They tell you what
> parameters you may want to modify, you know exactly what keyword to
> search for if you want more information on something.  And hopefully,
> the examples were set up by the experts, so they should be well
> configured for a purpose.  If the example also states what the purpose
> is (be it serving a database with 10,000 entries or a million entries)
> at least you have a starting point.
> 
> A mention (and reasonable example) in the quick start guide and a
> couple of examples along with warnings, common pitfalls, and pointers
> to the appropriate Berkeley documentation in the admin guide would go
> a long way to easing the learning curve.
> 
> P.S. - this faq entry: http://www.openldap.org/faq/data/cache/756.html
> says that "back-bdb, back-bdb and back-ldbm are "primary" storage
> database backends".  Is one of those supposed to be back-Hdb?
-- 
Michael L Torrie <torriem@chem.byu.edu>