Re: Optimizing OpenLDAP pam authentication (it's very slow)

[Matthew Gregg <greggmc@musc.edu>]

You are absolutely right, if OpenLDAP hopes to compete against NDS, AD
or for that matter other LDAP servers it will have to be very fast for
at least 10's of thousands of rows.  As it stands today, the
OpenLDAP. nsswitch, pam_ldap combination is too slow to be even be 
useful for authentication, unless i'm doing something wrong(which
could very well be correct).  I'm in the middle of trying to make this 
work as a campus authentication scheme and have gotten very
frustrated.  From the lack of documentation, it appears no one has
done this before.


A few more observations while fighting with openldap:

I note the same 100% cpu on a single user authenticating off the ldap
and this is running on a lightly loaded raid 5, dual PIII 450.
It really looks to be a problem with the group lookup.
Things grind to a halt after the password has been verified, with
this query(which can be seen by running "slapd -d 256"):
"(&(objectClass=posixGroup)(|(memberUid=greggmc)
(uniqueMember=uid=greggmc,ou=People,dc=musc,dc=edu)))"

If you use the migration scripts from PADL they will create group
ldif's that look similar to this:
dn: cn=itlab,ou=groups,dc=musc,dc=edu
objectClass: posixGroup
objectClass: top
cn: itlab
userPassword: {crypt}*
gidNumber: 1389
memberUid: binzafar
memberUid: jonesje
memberUid: sprovero
memberUid: starmerf
memberUid: starmerj

I noticed that the group query is using a different "membership"
objectclass of uniqueMember instead of memberUid. So I tried some
experiments.
I did a manual ldapsearch like this:
ldapsearch -H ldap://itlab.musc.edu -s sub -b "dc=musc,dc=edu"
"(&(objectClass=posixGroup)(|(memberUid=greggmc)
(uniqueMember=uid=greggmc,ou=People,dc=musc,dc=edu)))"

The query was very slow, just as it is during authentication.

Now if i change the query to this:
ldapsearch -H ldap://itlab.musc.edu -s sub -b "dc=musc,dc=edu"
"(&(objectClass=posixGroup)(|(memberUid=greggmc)
(memberUid=uid=greggmc,ou=People,dc=musc,dc=edu)))"

The query is very fast and still returns all the groups "greggmc"
belongs to.

So then I changed the  PADL migration scripts to create an ldif
like this:
dn: cn=itlab,ou=groups,dc=musc,dc=edu
objectClass: posixGroup
objectClass: top
cn: itlab
userPassword: {crypt}*
gidNumber: 1389
objectClass: groupofuniquenames
uniquemember: uid=binzafar, ou=People, dc=musc,dc=edu
uniquemember: uid=jonesje, ou=People, dc=musc,dc=edu
uniquemember: uid=sprovero, ou=People, dc=musc,dc=edu
uniquemember: uid=starmerf, ou=People, dc=musc,dc=edu
uniquemember: uid=starmerj, ou=People, dc=musc,dc=edu

I thought I was on to something, but alas, I reloaded the ldap and no
change in the group lookup speed.



On Wed, May 30, 2001 at 10:18:40PM -0600, Michael L Torrie wrote:
> On Wed, 30 May 2001, Matthew Gregg wrote:
> 
> > I have a similar performance problem myself however, I'm not using
> > kerberos.  I've also tried to get an answer here.  I say all this
> > hoping that you may find something I've missed.
> > If you do, please share.
> >
> > Here is what I have done:
> > Running slapd in debug (-d 256), it looks like the
> > performance hit occurs during the "group" lookup.
> >
> > I have the following indexes set:
> > index uid,cn,objectclass,uidnumber,gidnumber eq
> > index uniqueMember pres
> >
> > My LDAP contains ~12K users and ~13K groups.
> > Do you have a large LDAP store?
> >
> > Thanks.
> 
> You're probably right about the group thing.  I'll check that tomorrow.
> I only have about 2600 users in the database, and only about 40 groups.  I
> applied a simple index on uid, but that didn't help much at all.  I'll add
> indexes for gidnumber and the rest of the fields you've suggested and see
> what happens.
> 
> I'm pretty sure kerberos is not the problem because I can get a ticket
> very quickly using kinit.  The delay (as can be seen by top) is in the
> slapd itself.  The actual kerberos transaction is very quick.
> 
> Thanks for the ideas.  I'll try them and let you know how it works.
> OpenLDAP should be able to run much quicker, especially if it aims to
> compete with the likes of active directory or NDS.  a few thousand or even
> tens of thousands of entries shouldn't slow it down that much.  If there
> are a lot of transactions that cause the delay, then openldap should start
> caching information.  This would greately speed up logging out of windows,
> for example, when there are a ton of lookups on ownership, etc of all the
> files in the profile directory.
> 
> cheers,
> Michael
> 
> 

-- 
brought to you by, Matthew Gregg...
one of the friendly folks in the IT Lab.
--------------------------------------\
The IT Lab (http://www.itlab.musc.edu) \____________________
Probably the world's premier software development center.
Serving: Programming, Tools, Ice Cream, Seminars