Re: Tuning cachesize

[Quanah Gibson-Mount <quanah@zimbra.com>]

--On Wednesday, February 06, 2008 3:12 PM -0500 "Marantz, Roy" 
<Roy.Marantz@deshaw.com> wrote:

> Also, is it better (performance wise) to give more memory to the db
> cache or the entry or idl caches?  I couldn't find a clear answer from
> the Admin Guide, FAQ, nor the mailing list.
>
> Finally, is there any difference tuning bdb vs. hdb?  I would think not,
> but since I'm asking...

General guide to OpenLDAP tuning:

There are a number of parameters that will affect performance.  The ones 
with the most impact are:

BDB Cache size
cachesize
idlcachesize
threads
index


The first three of these settings have to do with the database.  The 
"threads" parameter affects the in/out rate of connections.  The "index" 
setting directly affects search performance.

Database tuning:

Of the three database paramemters, the BDB cache size is the most 
important.  There are two ways to tune for the BDB cachesize:

(a) BDB cache size necessary to load the database via slapadd in optimal 
time
(b) BDB cache size necessary to have a performant running slapd once the 
data is loaded

For (a), the optimal cachesize is the size of the entire database.  If you 
already have the database loaded, this is simply a du -c -h *.bdb in the 
directory containing the OpenLDAP data.

For (b), the optimal cachesize is just the size of the id2entry.bdb file, 
plus about 10% for growth.

The tuning of DB_CONFIG should be done for each BDB type database 
instantiated (back-bdb, back-hdb).


The slapd.conf "cachesize" setting:  The number of entries to store in the 
entry cache for slapd.  The most optimal value is of course, the entire 
number of entries in the database.  However, most LDAP servers don't 
consistently serve out their entire database, so setting this to a lesser 
number that more closely matches the believed working set of data is 
sufficient.  This is the second most important parameter for the DB.

The slapd.conf "idlcachesize" setting: Each IDL holds the search results 
from a given query, so the IDL cache will end up holding the most 
frequently requested search results.  For back-bdb, this is generally 
recommended to match the "cachesize" setting.  For back-hdb, this is 
generally recommended to be 3x"cachesize".



Slapd tuning:

"threads":  This directive sets the number of threads that slapd uses to 
process requests.   What this value should be I've generally found to be a 
function of the number of "real" cores on the system.  I.e., on a 2 CPU box 
with one core each, I set this to 8, or 4 threads per real core.  This is a 
"read" maximized value.  The more threads that are configured per core, the 
slower slapd responds for "read" operations.  On the flip side, it appears 
to handle write operations faster in a heavy write/low read scenario.

Search tuning:

"index":  This creates indices on various attributes in the database to 
speed up searching.  Know your searches, and configure indices 
appropriately.


--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration