[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: OpenLDAP system architecture?

Brad Knowles wrote:
On Thu, 2008-01-24 at 15:08 -0800, Howard Chu wrote: 
The single-master constraints on OpenLDAP were never about performance. Even
with OpenLDAP 2.2 the concurrent read/write rates for back-bdb are faster than
any other directory server. It's always been about data consistency, and the
fact that it's so easy to lose it in a multi-master setup.


I wasn't concerned so much about single-master versus multi-master.  I
was thinking more about the issue where a very read-intensive
environment did not mix well with an environment where a lot of writes
were also occurring directly to the system, but where bulk writes from
slurpd (at the time) would be handled relatively efficiently while also
handling high read intensity.

You seem to believe there was something magic about how slurpd propagated updates to slaves. There wasn't; slurpd was just a fairly stupid LDAP client that issued the same types of operations as any other client.

Thus the reason for ensuring that all updates went directly and only to
the master, the master was replicated out to the slaves via slurpd, and
the slaves only handled reads.


No, that wasn't the reason.

Of course, the modern architecture doesn't use slurpd, I was just
wondering if it might make more sense from a scalability perspective to
have a similar data flow architecture.

Not really. If you have a dedicated master that never has to answer any clients' read requests, you can get a slight performance boost for writes by disabling most indexing. The slaves will still have to index any writes they receive though, to provide good search performance, so on equal hardware the slaves will generally accept updates more slowly than they can arrive on the master.

You've been brainwashed by all the marketing lies other LDAP vendors tell
about multi-master replication. Multi-master has no relation to performance.


Again, I wasn't looking at single-master versus multi-master.  If I gave
that impression, I'm sorry.

It's only about fault tolerance and high availability. No matter whether you
choose a single-master or a multi-master setup, with the same number of
machines, the same number of writes must be propagated to all servers, so the
overall performance will be the same.


I'm confused.

So there's no performance benefit to doing bulk writes via syncrepl to
the slaves as opposed to individual writes to the master(s) via ldapadd?

There's no such thing as "bulk writes" in LDAP. Some vendors may have implemented their own proprietary bulk update mechanisms, but that's not standard LDAP.

Then why have syncrepl at all and instead just have everything handled
by ldapadd?


Huh?

I understand the consistency argument for single-master versus
multi-master, I'm just trying to find a way to partition the problem
space for performance reasons, in addition to any consistency reasons.

That's a pointless question. The right question is - how fast do you need it
to be? What load are you experiencing now, what constitutes a noticeable
delay, and how often do you see those?


Good questions, but I'm not sure I've got the answers.  I know that our
OpenLDAP directory system is going to be used as a critical component of
a campus-wide authentication system, and the target for the
authentication system is to handle at least hundreds of authentications
per second.  Problem is, I don't know what that translates to in terms
of numbers of OpenLDAP operations per second, and what the mix of reads
are relative to the writes.

Hundreds = trivial. When you get to *hundreds of thousands per second*, then you'll have a problem, because that will require multiples of gigabit network bandwidth. Even so, OpenLDAP can handle it.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/