Re: multiple attribute search with single return slowness

[Tony Earnshaw <tonye@billy.demon.nl>]

man, 26.04.2004 kl. 05.16 skrev Blomberg David:

> I am having a slowness response problem using Postfix on OpenLDAP (I
> have narrowed it down to LDAP response time being my largest bottleneck
> when querying multiple attributes.
> (This system supports over 4000 user accounts so before telling me how
> fast it is for you I noticed a big speed didference between 2000 users
> which was quick and when I went up to 4000---there might be something I
> misssed or a change I need to make for the larger user base if anyone
> knows of this please let me know)

I have standardized on RH RHEL3/RHAS3, Openldap 2.2.11, Postfix
2.0.19-20040312 (soon to be 2.1, just released), Sleepycat BDB 4.2.52,
Openssl 0.9.7d and Cyrus SASL 2.1.18 for production. Performance is
always more than satisfactory. I don't have anything like the number of
users you do at any site, and can only offer comments on points that I
regard as important.

> for instance:
> 
> (&(|(mail=%s)(mailalternateaddress=%s))(accountstatus=active))
> which I ask to return "mailmessagestore"
> 
> and 
> 
> (|(mail=%s)(mailalternateaddress=%s))
> which I have return "mailforwardingaddress"
> 
> System:
> SMP Xeon based system with 4 GB RAM running Red Hat Enterprise Server
> 3.0 using Postfix (fixed version that works against LDAP) and OpenLDAP
> as provided by Red Hat.

This is OL 2.0.27, the alternative offered by RH is, AFAIK 2.1.22 which
is buggy and should not be used. Use the latest source, or 3rd party
rpms of the latest source. I always compile my own, versions as detailed
above. Keep source compiles in /usr/local, apart from Cyrus SASL
updates, which will have to go in /usr/lib/sasl2 (usually a symlink) but
won't ruin anything :)

>  (I am coming here as Red Hat support is all but
> non-existant and I am tring a custom compiled from Open LDAP version in
> parallel to the full Red Hat system).

Always keep up to date with thi stuff, where possible.

> What I have tried up until now:
> 1-enabling Postfix side LDAP caching 
> 	ldapXXXX_cache = yes
> 	ldapXXXX_cache_expiry = 60000
> 	ldapXXXX_cache_size = 32768

LDAP caching is not supported any more; Postfix hasn't supported it for
a long while. Postfix does support connection pooling, whice speeds
things up. So does using ldapi (unix domain sockets) on localhost, with
recent Postfix snapshots and 2.1.

Do not rely on on slapd.conf memory caching, either. Use a DB_CONFIG
configuration file in $prefix/var/openldap-data and configure it for a
large enough RAM cache. Sleepycat's docs explaining this are included in
the BDB source tarball. Also, especially Quanah Gibson-Mount
(stanford.edu) on this list has quite often detailed this - search the
archives.

> 	This did not work at all it only resulted in a number of errors to the
> log and a web search that blamed it not working of LDAP..... (Actually I
> believe this would fall to Postfix, but regardless it is broken so I am
> unable to cache the results
> 
> 2-I started the search lower on the tree before I searched in
> "o=domain.com" now I search in "ou=local_people,o=domain.com" and
> "ou=remote_people,o=domain.com"  
> 
> 	This didnt show any speed ups.
> 
> 3-I tried downloading and compiling OpenLDAP from scratch.
> 	After fully populating the database I find it a little quicker and on
> various test I come up as twice as fast for single attribute look ups,
> but only marginally faster for multiple attribute lookups.
> 	(As an aside I found that Red Hat was allowing some bad data to be
> input that OpenLDAP strict checking does not allow but Red Hat LDIF
> import never complaigned about so I fixed the entries-if nothing else at
> least I got some mess cleaned out of the system)

Good, you got past that hurdle ;)

> 4-I tried adjusting my indexes in slapd.conf:
> 	index
> mail,mailalternateaddress,accountstatus,mailmessagestore,mailforwardingaddress eq
> 	Unfortunately after doing this and rebuilding the indexes I started
> loosing the ability to log in and other odd errros developed until I
> undid this.
> **actaully I am hoping someone can tell what I messed up on this ont as
> proper indexes are my best hope.**

You definitely need indices if you want performance. There's no reason
that creating correct indices should give the results you describe. It
may well be that you have a bad compile (cross-linked library versions,
openssl, whatever). Try with your own source of all of the above - I've
never had what you describe. Going through the slapd log at -d 265 will
tell you what indices you've done badly, f.ex. <=
bdb_equality_candidates: (uniqueMember) index_param failed (18) will
tell you you should have an equality index for uniqueMember.

> 5-Disabled the binding and all look up are now annonymous
> 	slight performance increase.

Since this means slackened security, it isn't a good idea.

> Any help that could help me increase the LDAP response speed would be
> very much appreciated.

Quanah is servicing thousands of Stanford e-mail users, though on
Solaris and with Sendmail, so it's possible.

Also, I have Postfix local(8) and virtual(8) hand off mail to LDAP-based
maildrop (1.6.3) for delivery, which also speeds things up a bit.

HTH,

--Tonni

-- 

We make out of the quarrel with others rhetoric
but out of the quarrel with ourselves, poetry.

mail: billy - at - billy.demon.nl
http://www.billy.demon.nl