RE: how scalable is openldap

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of asr@ufl.edu

> You will run into size limitations for some of the things you want to stick
> into this size of a database.  My group which records every
'authenticatable
> user' is larger than 8MB, I was dumping core when I tried to ldapadd or
> ldapmodify it, Howard recommended a compilation tweak which we've yet to
> attempt.

A fix for this was in release 2.1.15. There was a bug in that fix, which
itself was fixed in 2.1.16. This should no longer be a problem, as long as
your data fits inside the sockbuf_max_incoming limits.

> You will run into size limitations on the database containers: id2entry.dbb
is
> WAY over 2G.  If your filesystem gets mad at that you've got problems.

Yah, you need a filesystem that supports 64-bit sizes and offsets...

> You will run into locking limitations on modifications of some large
entries.
> I haven't been able to elicit the problem on demand, but I fairly regularly
> find that OpenLDAP stops responding to queries while it processes a modify
for
> a smaller, but still large, indexed group (60,000 members or so)

Stops? Does it just hang forever, or does it eventually come back? If it
comes back, it's probably just overhead from writing the transaction log.
Make sure the log is on a separate disk from the database files. Any large
write that ties up a lot of data like this has the potential to get in
everything else's way, but it should only be temporary.
>
> I still "occasionally" get what I'd refer to as database corruption:
> ldapdelete sez "It's not here", ldapadd sez "It's already here".

That's pretty odd. However, the DN2ID index that OpenLDAP currently uses is a
definite weak point in the scalability department. This problem will be
addressed in a future release.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support