Re: scaling openldap for performance

[Trevor Warren <trevorwarren@yahoo.com>]

Hello All,

 My results for the 20/50/100 user tests are as
follows:

>> OpenLDAP  scales *Wonderfully* well and peaks at
about 300+ reads per sec on our hp bl20p blade. The
config file is as below and has been used to cache
about 10,000 odd entries DB_CONFIG file as below.

>> Without any tuning of the DB Backend i would obtain
about 300+ reads/sec wrt read only transactions. 

>> With DB Tuning using huge DB Caches as in the
DB_Config below i did obtain 150+ writes/sec as
compared to 15+ reads/sec i did earlier.

>> The whole setup is till *Very unstable*. Probably
the params i did put in are causing it to behave so.
Once for a 20user test with Zero Think Time openldap
went down giving a malloc error. 

>> On another occasion with a 50 user test and Zero
Think Time OpenLdap goes down with a segfault. When i
try to end the process with a ctrl+c there seems to no
response and the slapd process keeps messing around in
memory dunnow doing something crazy. When i kill -9 it
the db goes unconsistent. How the hell can it take
greater than 5mins just to abort all tasks an end
ceremoniously.

>> I can separate the logs onto another disk since i
have only one set of SAN Disk partitions.

>> Openldap seems to scale well with the DB config as
i performen below. But then on write loads of 20+
users it simple goes for a toss.

>> Do we still have a lot of work to go in towards
being able to scale openldap from here????. Or you all
still think it is possible to make it perform in the
env i am testing it in.

 Remember i  have a 30million user base. Reads will be
about 90% of the time while write + updates < 5-7% of
the time. But this fellow should scale to atleast 80%
of that. What say???

###########################
#slapd.conf
###########################
#
# Base Database Type and Domain
#
database        bdb
suffix          "dc=somedomain,dc=com"

#
# Distinguished Name allowed  complete
# access to database backend.
#
# This section should be removed after
# the   initial  databases  have  been
# created and loaded.
#
rootdn          "cn=Manager,dc=somedomain,dc=com"
rootpw          {CRYPT}<some crypt password>

#
# Database Directory
directory       /opt/openldap/var/bdb-data

#
# Set the entry cache size to 5000.
#
cachesize       10000

#
# Set transactional checkpoint.
#       
checkpoint      2000     10

#######################
#DB_CONFIG file
#######################

#
	# Set the database in memory cache size.
	#
	set_cachesize   0       52428800        0
	
	
	#
	# Set database flags.
	#		
	set_flags       DB_TXN_NOSYNC
	
	
	#
	# Set log values.
	#
	set_lg_regionmax        200000000 
	set_lg_max              10485760
	set_lg_bsize            2097152
	set_lg_dir              /opt/openldap/var/bdb-data
		
	#
	# Set temporary file creation directory.
	#			
	set_tmp_dir             /tmp

 The DB cache stands at close to 200MB.

Trevor

=====
( >-                                           -< )
/~\    ______________________________________   /~\
|  \) /    Scaling FLOSS in the Enterprise   \ (/ |
|_|_  \        trevorwarren@yahoo.com        / _|_|
       \____________________________________/


	
		
__________________________________
Do you Yahoo!?
Friends.  Fun.  Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/