Re: OpenLDAP system architecture?

["Gavin Henry" <ghenry@suretecsystems.com>]

<quote who="Quanah Gibson-Mount">
> --On Thursday, January 24, 2008 12:12 PM -0600 Brad Knowles
> <b.knowles@its.utexas.edu> wrote:
>
>
>> I do not yet understand a great deal about how our existing OpenLDAP
>> systems are designed, but I am curious to learn what kinds of
>> recommendations you folks would have for a large scale system like this.
>
> This is generally good information to know...
>
> But basically, have you read over the information on understanding your
> system requirements?  I.e., how to properly tune DB_CONFIG and slapd.conf?
>
>> In the far, dark, distant past, I know that OpenLDAP did not handle
>> situations well when you had both updates and reads occurring on the
>> same system, so the recommendation at the time was to make all updates
>> on the master server, then replicate that out to the slaves where all
>> the read operations would occur.  You could even go so far as to set up
>> slaves on pretty much every single major client machine, for maximum
>> distribution and replication of the data, and maximum scalability of the
>> overall LDAP system.
>
> Updates -> master is always recommended.  You can set up multi-master with
> 2.4, but it will be slower than a single master scenario.  The general
> best
> practice for fail over is to have a primary master that receives writes,
> and a secondary master that is getting the updates, and will take over via
> fail-over mechanisms if the primary goes down, becoming the new primary.
>
>> If you did use a multi-master cluster pair environment that handled all
>> the updates and all the LDAP queries that were generated, what kind of
>> performance do you think you should reasonably be able to get with the
>> latest version of 2.4.whatever on high-end hardware, and what kind of
>> hardware would you consider to be "high-end" for that environment?  Is
>> CPU more important, or RAM, or disk space/latency?
>
> RAM is probably the most important, but you also will want fast disks,
> proper partitioning of the logs separate from the database and logs, and I
> recommend a non-journaling filesystem.  2 or more cores is also useful.
> Unfortunately I don't really see enough information from your end (yet) to
> really say much beyond that.
>
>> Alternatively, if you went to a three-level master(s)->proxies->slaves
>> architecture [0], what kind of performance would you expect to be able
>> to get, and how many machines would you expect that to be able to scale
>> to?  Are there any other major issues to be concerned about with that
>> kind of architecture, like latency of updates getting pushed out to the
>> leaf-node slaves?
>
> On the SunFire x4100 servers I used to have, I could easily obtain some
> 23,000+ reads/second with OpenLDAP 2.3 on a single server.

See:

http://www.connexitor.com/blog/pivot/entry.php?id=191#body
http://www.openldap.org/doc/admin24/replication.html#MirrorMode

>
>> How about the ultimate maximum distribution scenario, where you put an
>> LDAP slave on virtually every major LDAP client machine?
>
> Seems like major overkill to me, unless you are getting hundreds of
> thousands of reads/second.
>
> --Quanah
>
>
> --
>
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> --------------------
> Zimbra ::  the leader in open source messaging and collaboration
>