Re: OpenLDAP system architecture?

[Quanah Gibson-Mount <quanah@zimbra.com>]

--On Thursday, January 24, 2008 12:12 PM -0600 Brad Knowles 
<b.knowles@its.utexas.edu> wrote:


> I do not yet understand a great deal about how our existing OpenLDAP
> systems are designed, but I am curious to learn what kinds of
> recommendations you folks would have for a large scale system like this.

This is generally good information to know...

But basically, have you read over the information on understanding your 
system requirements?  I.e., how to properly tune DB_CONFIG and slapd.conf?

> In the far, dark, distant past, I know that OpenLDAP did not handle
> situations well when you had both updates and reads occurring on the
> same system, so the recommendation at the time was to make all updates
> on the master server, then replicate that out to the slaves where all
> the read operations would occur.  You could even go so far as to set up
> slaves on pretty much every single major client machine, for maximum
> distribution and replication of the data, and maximum scalability of the
> overall LDAP system.

Updates -> master is always recommended.  You can set up multi-master with 
2.4, but it will be slower than a single master scenario.  The general best 
practice for fail over is to have a primary master that receives writes, 
and a secondary master that is getting the updates, and will take over via 
fail-over mechanisms if the primary goes down, becoming the new primary.

> If you did use a multi-master cluster pair environment that handled all
> the updates and all the LDAP queries that were generated, what kind of
> performance do you think you should reasonably be able to get with the
> latest version of 2.4.whatever on high-end hardware, and what kind of
> hardware would you consider to be "high-end" for that environment?  Is
> CPU more important, or RAM, or disk space/latency?

RAM is probably the most important, but you also will want fast disks, 
proper partitioning of the logs separate from the database and logs, and I 
recommend a non-journaling filesystem.  2 or more cores is also useful. 
Unfortunately I don't really see enough information from your end (yet) to 
really say much beyond that.

> Alternatively, if you went to a three-level master(s)->proxies->slaves
> architecture [0], what kind of performance would you expect to be able
> to get, and how many machines would you expect that to be able to scale
> to?  Are there any other major issues to be concerned about with that
> kind of architecture, like latency of updates getting pushed out to the
> leaf-node slaves?

On the SunFire x4100 servers I used to have, I could easily obtain some 
23,000+ reads/second with OpenLDAP 2.3 on a single server.

> How about the ultimate maximum distribution scenario, where you put an
> LDAP slave on virtually every major LDAP client machine?

Seems like major overkill to me, unless you are getting hundreds of 
thousands of reads/second.

--Quanah


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration