Re: OpenLDAP's backend for performance and high reliability

[Buchan Milne <bgmilne@staff.telkomsa.net>]

On Monday 15 October 2007 02:36:39 Tommy Pham wrote:
> My concerns are not just about performance for 1 box setup or 1 master
> with multiple slave replications and proxies.  I'm more interested in
> the robustness such as Dynamic Schema(s), Multi-Master Replication, and
> Dynamic configuration (as featured in Apache DS).

Howard answered you on these aspects.

> Multi-master or 
> cluster setup have higher reliability and performance under heavy load
> with large data in my experience.

Multi-master's only real benefit is a cheaper "HA cluster" feature. HA 
clusters are not necessarily more reliable (more complex), and don't perform 
any better (unless this is purely because the storage backend is faster ...). 
No cluster/multi-master setup is going to help with write load, and read load 
can easily be spread with slaves.

> Also, because I'm migrating from MS 
> based platform, I intend to integrate other application servers into
> LDAP as well such as DNS (via bind-dlz),

bind-dlz supports LDAP (but I won't believe the performance results at 
http://bind-dlz.sourceforge.net/ldap_perf.html myself - especially since no 
versions are listed, and the entry cache seems to be about 1000 times too 
small), but I note that, unless you are doing mass DNS hosting (hundreds of 
zones), bind sdb_ldap supports DNS records in LDAP sufficiently IMHO (I have 
about 10 zones stored in OpenLDAP).

> FTP

All popular Unix FTP servers support LDAP, for authentication at least, most 
support other features (such as Bandwidth quotas etc.).

> , e-mail & groupware,

Most Unix groupware suites (zimbra, scalix, kolab, insight, etc.) require or 
ship OpenLDAP, and use the directory as their primary account store.

> Samba,

The only supported non-local-file password backend for Samba is LDAP. OpenLDAP 
is probably the most commonly used LDAP server with Samba.

> etc... in the same way as MS integrates DNS and Exchange in it's 
> Active Directory.  Will OpenLDAP with back-bdb/hdb support all of that
> and still perform well when there are over millions of entries?

Why not ? It supports our mail system with 1.1 million entries. We don't have 
that much in the way of samba/DNS/dhcp/sudo/freeradius entries, only a couple 
of hundred of each, but I don't see how the kind of account is relevant. Only 
the attribute size/index and query loads are relevant.

> As for 
> native DB support vs layer like ODBC, why not just use the DB's native
> client library?  (I guess this falls in line with development mailing
> list more than this mailing list.)

I'm guessing (with some experience in high-load environments using ODBC for 
things such as bandwidth accounting databases) the ODBC layer is probably not 
the bottleneck ... so why sacrifice portability (e.g. users using 
distributions shipping OpenLDAP with odbc support should in theory only need 
to install the Oracle client software to be able to use Oracle) for a small 
performance gain.

> I understand that "a directory is a 
> specialized database optimized for reading, browsing and searching" and
> not writing.  That's why I opt for having dedicated RDBMS vs embedded
> for distributed computing...

Maybe you should rather opt for having a dedicated directory server accessed 
via the LDAP protocol instead of some embedded database (I don't know what 
you are referring to here ... it can't be OpenLDAP ...)?

I note that replication features in OpenLDAP are most likely superior in 
flexibility/robustness to most popular RDBMSs (depending on your requirements 
of course).

> just as enterprise applications are 
> developed in n-tier.

And OpenLDAP is a very reliable implementation of LDAP as one of those 
enterprise tiers, and used as such in many organisations.

Regards,
Buchan