[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: FW: OpenLDAP as an enterprise level LDAP provider (try II)

John,

You really need something that formats your email better.

First some notes:

slapadd indexes while adding the DB.

First, There is no reason to index *after* slapadd-ing, you are wasting your time.

Second, you don't say what hardware platform you are using and what the memory/disk resources are. OpenLDAP (and all directory services from what I can tell) is bounded by memory and disk. You need plenty of both.

Third, as I recall, you can hot backup BDB starting with the 4.2 branch. I personally use the slapcat option.

Fourth, you do not say what BDB flags you enable/disable when slapadding. This can greatly impact your slapadd time.

Fifth, you do not say how large you configured your BDB DB Cache to be. This can greatly impact your slapadd time.

Finally, I'll note that it took me 16 minutes to slapadd (with indexing) a large 500k entry DB on a Dual CPU Dell 1750 with 2GB of RAM. 2GB of RAM was not sufficient however, to hold the DB in memory as well as run slapd. I'd guestimate the system would truly need 4GB of RAM for that.

Regards,
Quanah

--On Wednesday, November 24, 2004 12:55 PM -0600 "Fortin, John {PBG}" <John.Fortin@pepsi.com> wrote:

Looks like this may = have been cut off on the first post. John Fortin
PBG Middleware and Web = Services (914) 767-7844 > -----Original
Message----- >From: Fortin, John {PBG} >Sent: Wednesday, November 24,
2004 12:21 PM >To: OpenLDAP Mail List >Subject: OpenLDAP as an enterprise
level LDAP = provider > >First of this, this message is = intended to
open a discussion >about using OpenLDAP in the = enterprise. I do not
want to >start a flame war concerning the = pros and cons of various LDAP
>implementations. > >Currently we are using OpenLDAP = as our initial
implementation >for authentication and = authorization with Weblogic and
other >J2EE providers for our enterprise = application. Our initial
>rollout was successful, although = we did not have a large >population
of users in the = directory (fine, and we had no issue with = loading
data etc as the ldif >files were small. > >However, as we are now looking
to = roll this out to a much >larger population (600K+) we are = starting
to run into some >issues, one of which I sent a = note about recently.
The issues >we are currently seeing, and = could potentially be a show
>stopper for us are as = follows: > > 1) Log = archiving and transactions
- With the current >bdb and version of OpenLDAP = (2.2.18), I cannot
archive/delete >files without stopping = slapd. This manifested itself as
we >were testing bulk loading of data = and consistently ran out of >log
space. I have tested = with the various patched suggested >to no avail. I
have not = tested with the newest version of bdb >(4.3) as I have no
indication = that this fixes the issue. > >   2) The ability to backup
data - Using the bdb utilities >(db_load and db_dump) do not = work. It
seems that this is >based on OpenLDAP using custom = hashes in the
creation of the >configured indexes. (This is = based on some discussion
I found >in the maillist archives). = The two workarounds that I am
>aware of, creating ldif files = with slapcat, and backing up the >bdb
files themselves so not seem = to be adequate for the >following reasons:
> = >* slapadd - with 600K users and no = indexes it takes about 2 > hrs
to load. The = creation of indexes afterwards with > slapindex takes an =
additional 6-12 hours. To me, this seems > like too long of a time = for
recovery. >* *.bdb file backup - we've had = limited success with this. >
There also seems to be an = issue, even after doing a > db_checkpoint and
a = db_recover of a dependency on logs > files. As we are = looking to do
a 'cold' backup of our > master ldap directory, we = do not want to be
dependent on logs files. > >I have searched the archives = quite a bit
looking for similar >issues with limited = success. I know the basics of
how >OpenLDAP works and tuning of the = system, but I am by no means >a
guro in the internals. At = this point, I am looking for some >direction
as to how to = proceed. > >System: >* OS: RH ES 3.0 >* OpenLDAP 2.2.18 >*
BDB 4.2.52 (with current = patches) > >Thanks!! >--John > >John Fortin
>PBG Middleware and Web = Services >(914) 767-7844 >




--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html