Re: OpenLDAP and BIG mail environments

[Syahrul Sazli Shaharir <sazli@surfopen.com>]

On Wed, 11 Feb 2004, James Courtney wrote:

> Is anyone working, running, or has experience with large (> 100,000)
> user mail systems, particularly those making heavy use of IMAP and using
> components like Postfix, Courier-IMAP, Maildrop, etc.

*raises hand*

In one site: running sendmail+maildrop+courier-imap (all querying LDAP for
routing+delivery+POP&IMAP), for > 100,000 users (a lot more actually --
can't disclose here) for over 2 years now. sendmail+maildrop queries 3
slaves, courier-imap queries 3 more, all load balanced via L4 switch. This
particular site is a bit behind in versions though (still running 2.0.x
line), but no serious problems so far...

> We're working on a Postfix/Courier-IMAP setup backed by our OpenLDAP
> user database but we're very IMAP dependent and will likely need to
> support several hundred thousand users.

In our experience, IMAP won't be taxing on your LDAP so much (even though
we're running webmail which authenticates on every click -- currently
testing imap proxy/cacheing tools to avoid this) -- bottlenecks we hit so
far are all related to our maildir storage I/O.

> I'd love to pick someone's brain about any potential issues with
> OpenLDAP with larger data sets and also the more general topic of
> hanlding a mail system that large.  Feel free to contact me off group if
> you have mail server stuff advice that isn't OpenLDAP relevant too.

The most important thing IMHO is to make sure that your LDAP applications'
query patterns are sufficiently normalized, i.e. avoid / defend from
spikes in querying rates. Examples:-

1. IMAP server: delay on each failed login attempt: this is to avoid brute
force attacks from bashing your LDAP servers. (easy for courier-imap, it
has built-in 2-second delay).

2. MX/sendmail server: protect against spammer dictionary attacks,
mailbombs or similar. This is a lot harder, in fact most of our defense
mechanisms at this point are reactionary (block manually upon event).
The only preventive ones are making use of blocklists (esp. block dynamic
IPs).  Anyone wish to share better ways? (oops, off topic, pls mail me &
the original poster off list).

Other than that, search through this list for the usual suggestions on db
cache/memory/indexing/minimal ACL, etc. for your LDAP db.

Good luck! :)

--sazli
[ http://sazli.surfopen.com     |    cd /open/source; make world ]
[ http://pgp.mit.edu:11371/pks/lookup?search=0x382141B4&op=index ]