open LDAP query slow for 5k users only (each having 40 entries)

[Vishal Mathur <vishal@netyantra.com>]

Hi all,

I am using open ldap 2.0.23 server from last 3 years on my linux server 
(Pentium(R) 4 CPU 2.80GHz) or storing and retriving user profiles. It 
works fine for arround 100 to 200 user profiles. Once number of entries 
increase to 5 k, It took arround 1.5 second to add one users in ldap. I 
was surprise to see that once i added arround 5000 profiles in system 
searching of a profile started taking .15 seconds. This time does not 
include TCP connect time as my C client opens a connection at startup.

My ldap tree look like =>
rootdn=mydomain
 |
 =>pid=101@example.com (arround 40 attributes)
         |
          => property=1,101@example.com (arround 10 attributes)
              |
              => property=1,101@example.com (arround 10 attributes)
          => property=2,101@example.com (arround 10 attributes)
          .
          . (arround 40 sub entries under one users)
 =>pid=102@example.com
         |
          => property=1,102@example.com (arround 10 attributes)
          => property=2,102@example.com (arround 10 attributes)
          .
          . (arround 40 sub entries under one users)
 .
 .
 =>pid=5000@example.com

So i have arround 50 k users profiles each having arround 39 entries under
user tree.

I am trying search with ldap_search_ext with
  dn => pid=101@example.com,dc=nycube
  filter => NULL
  scope => sub
timelimt and sizelimit is also NULL.

I tried duing indexing on object class. But that has solved down the search
also. I did slapindex after changing slapd.conf.

I must be missing some important conf things that why it is so slow.
Please guide me to configure and use it for 50 users.

Please find the attached slapd.conf and myldapschema.schema file for 
referance.

--
Vishal Mathur
Manager - Applications Group
Netyantra India Pvt. Ltd.
3rd Floor, Jaysynth Center, Plot No. 6,
Sector 24, Turbhe, Mumbai - 400 705
Phone: 91 22 27832547/9

-------------------------------------------------------------------------------
Disclaimer -
If you are not the intended recipient of this transmission to whom it is
addressed, or have received this transmission in error, you are hereby
notified that any dissemination, distribution or copying of this transmission
is strictly prohibited. Please notify us immediately and delete this e-mail
from your system. The sender does not accept liability for any errors or
omissions in the contents of this message which arise as a result of e-mail
transmission, which cannot be guaranteed to be secure or error-free as
information could be intercepted, corrupted, lost, destroyed, arrive late or
incomplete, arrive at wrong address or contain viruses. If verification
is required please request a hard-copy version.  This e-mail contains only the
personal opinions of the sender and does not represent an official
communication from NetYantra of any manner.
-------------------------------------------------------------------------------


#defination for attributes (for all objects)

#set of attributes for profile object class 
attributetype ( 10.1.1 NAME 'pid' SUP name SINGLE-VALUE )
attributetype ( 10.1.2 NAME 'leftHours' SUP name SINGLE-VALUE )
attributetype ( 10.1.3 NAME 'leftMinutes' SUP name SINGLE-VALUE )
attributetype ( 10.1.4 NAME 'password' SUP name)
attributetype ( 10.1.7 NAME 'selfIp' SUP name)
attributetype ( 10.1.8 NAME 'selfPort' SUP name SINGLE-VALUE )

attributetype ( 10.1.9 NAME 'type' SUP name SINGLE-VALUE )
attributetype ( 10.1.10 NAME 'permission' SUP name SINGLE-VALUE )
attributetype ( 10.1.11 NAME 'maxCall' SUP name SINGLE-VALUE ) ##don't know how to use it as integer ??
#### TOTAL ARROUND 100 Attributes

#defination of Object classes 
#Object class profile
objectclass ( 10.2.1 NAME 'profile' SUP top STRUCTURAL
	MUST ( pid $ userPassword )
	MAY  (hiddenUser $ UserType $ chargeType $ AliasOfUser $ AliasType $ displayName))	

#Object class user_profile
objectclass ( 10.2.2 NAME 'uP' SUP top STRUCTURAL
	MUST ( pid $ pT )	
	MAY  ( userPassword $ dtmf $
           selfIp $ selfPort $ 	ivrsNumber ))
	
###Arround 50 object classes

# This file should NOT be world readable.
include		core.schema
include 	myldapprofile.schema

pidfile		/usr/local/var/slapd.pid
argsfile	/usr/local/var/slapd.args

loglevel 0

access to dn=".*,dc=example.com"
  	by dn=".*,dc=example.com" write
	by dn="cn=manager,dc=example.com" write
	by anonymous auth
	by * none

access to * 
	by dn="cn=manager,dc=example.com" write
  	by dn=".*,type=admin,dc=example.com" write
	by anonymous auth
	by * none

defaultaccess none

database	ldbm
suffix		"dc=example.com"
rootdn		"cn=manager, dc=example.com"

cachesize 10000
dbcachesize 1000000
sizelimit 10000

#### Tried both with indexing and without indexing. Executed slapindex after changes
#index objectClass pres,eq

rootpw		809kmm2ji3jnm9u829e4