RE: OpenLDAP as an enterprise level LDAP provider

["Matthew Hardin" <mhardin@symas.com>]

The message was pretty badly mangled, but mostly complete. Maybe post in
plain text next time?

Our customers routinely use our distributions of OpenLDAP, called Connexitor
Directory Services, in enterprise directory applications with a great deal
of success.

I'll address your issues one at a time:

1. Log archiving and deleting - Because of a nasty deadlocking problem, the
author of back-bdb/back-hdb (our own Howard Chu) needed to make a change to
those backends that created a long-lived transaction. This transaction is
what prevents log files from being rotated/deleted. He produced a patch to
Berkeley DB 4.2 that would address this problem, but it's not an official
Sleepycat patch and so you probably do not have it. It is available
publicly, though, and our distributions are built with this it. In version
4.3 Sleepycat addressed the functionality introduced by the patch, but to
make use of it you will need updates that are currently only in
OPENLDAP-RELEASE. Your alternatives are to live with the problem, patch and
rebuild Berkeley DB and OpenLDAP, or use Berkeley DB 4.3.

2. The ability to back up data - 
a) You are correct: Don't use the Berkeley native tools. Also, to insulate
you from changes to the internal database, don't back up the database files
themselves; use ldapsearch or slapcat to dump the db in LDIF format. Because
of the fine-grained locking and the transactional nature of the bdb
backends, slapcat can be used while slapd is running.

b) The load times you are complaining about can be reduced through tuning
and setup changes. How often do you anticipate reloading the DB? I hope not
very often...

c) You are best served by letting slapadd create the indexes when it creates
the new db. It's much faster.

Hope this helps...

Matthew Hardin
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
http://www.symas.com

-----Original Message-----
From: owner-openldap-software@OpenLDAP.org
[mailto:owner-openldap-software@OpenLDAP.org] On Behalf Of Fortin, John
{PBG}
Sent: Wednesday, November 24, 2004 9:21 AM
To: OpenLDAP Mail List
Subject: OpenLDAP as an enterprise level LDAP provider

First of this, this message is =ntended to open a discussion about using
OpenLDAP in the =nterprise. I do not want to start a flame war concerning
the =ros and cons of various LDAP implementations. Currently we are using
OpenLDAP as our =nitial implementation for authentication and authorization
with =eblogic and other J2EE providers for our enterprise application. =ur
initial rollout was successful, although we did not have a large =opulation
of users in the directory (<1000) Performance was =ine, and we had no issue
with loading data etc as the ldif files were =mall. However, as we are now
looking to roll =his out to a much larger population (600K+) we are starting
to run =nto some issues, one of which I sent a note about recently. The
=ssues we are currently seeing, and could potentially be a show stopper =or
us are as follows: 1) Log archiving and transactions - With the =urrent bdb
and version of OpenLDAP (2.2.18), I cannot =rchive/delete files without
stopping slapd. This manifested =tself as we were testing bulk loading of
data and consistently ran out =f log space. I have tested with the various
patched suggested to =o avail. I have not tested with the newest version of
bdb (4.3) =s I have no indication that this fixes the issue. 2) The ability
to backup data - Using the bdb utilities =db_load and db_dump) do not work.
It seems that this is based on =penLDAP using custom hashes in the creation
of the configured indexes. =This is based on some discussion I found in the
maillist =rchives). The two workarounds that I am aware of, creating ldif
=iles with slapcat, and backing up the bdb files themselves so not seem =o
be adequate for the following reasons: = slapadd - with 600K users and no
=ndexes it takes about 2 hrs to load. The creation of indexes =fterwards
with slapindex takes an additional 6-12 hours. To me, =his seems like too
long of a time for recovery. = *.bdb file backup - we've had limited =uccess
with this. There also seems to be an issue, even after =oing a db_checkpoint
and a db_recover of a dependency on logs =iles. As we are looking to do a
'cold' backup of our master ldap =irectory, we do not want to be dependent
on logs files. I have searched the archives quite a =it looking for similar
issues with limited success. I know the =asics of how OpenLDAP works and
tuning of the system, but I am by no =eans a guro in the internals. At this
point, I am looking for =ome direction as to how to proceed. System: OS: RH
ES 3.0 OpenLDAP 2.2.18 BDB 4.2.52 (with current =atches) Thanks!! --John
John Fortin PBG Middleware and Web =ervices (914) 767-7844