Re: slapd-bdb cachesize

[Igor Brezac <igor@ipass.net>]

On Wed, 12 Mar 2003, Tino Lange wrote:

> Igor Brezac wrote:
>
> > I use openldap 2.1.15/bdb-4.1.25 (dbd backend type) on Solaris 9 system
> > with 512MB of RAM.  The slapd process continues to grow and I'd like to
> > know if this is normal based on my configuration parameters.  I am unclear
> > as to how does cachesize affect memory usage.  Do you multiply this number
> > with the average entry size (and the number of threads?)?  Is this param
> > neccessary to set in addition to DB_CONFIG tuning?
>
> Hi Igor!
>
> Good to hear someone has the same problem.
> I also realized that my slapd is constantly growing, to IMHO rather
> unexpected sizes.
>
> But: despite questioning this list unfortunately noone answered so far,
> whether this is normal behaviour or not
> (See my postings: "BDB log.xxx Files" thread started 26.02.2003 17:25
> and later seperately as "slapd memory usage (was: Re: BDB log.xxx
> Files)" started 28.02.2003 15:57)
>
> In those mails you can find details about my setup.
> My slapd's start with about 2.5 MB and grow by time slowly quite some MB
> a day (in the first days faster of course, this seems to be the normal
> caching). When it reaches about 300 MB I restart the process (which is
> much more than one MB for each entry in the database!!!)
>
> Meanwhile I tried inserting the checkpoint option in slapd.conf to avoid
> also using lots of diskspace, this works fine so far, but doesn't seem
> to reduce the memory hunger.
>
> I also recompiled OpenLDAP with the new BDB related patch for Berkeley
> DB (4.1.25.1) - but this doesn't seem to help, too.
>

I applied the patch as well.

> Could you please give some more information about your setup, your
> database, your indices, your usage - so we can compare and collect this
> information? Maybe the others on the list can give us more hints then, -
> the more they know the better they might help us, hopefully.
>

# Global
threads         12

# Backend - o=pb
database                bdb
cachesize               10000
checkpoint              0 30

index                   objectClass             eq
index                   cn                      eq
index                   sn                      eq
index                   seeAlso                 eq
index                   uid                     eq
index                   uidNumber               eq
index                   gidNumber               eq
index                   memberuid               eq
index                   member                  eq
index                   pbNumber                eq
index                   sendmailMTAKey          eq
index                   sendmailMTACluster      eq
index                   sendmailMTAHost         eq
index                   sendmailMTAClassName    eq
index                   associatedDomain        eq

I have about 45,000 entries and will grow to about 100,000.  A fairly
straight forward setup, but busy usage.

In my DB_CONFIG I have
set_cachesize 0 100663296 2

This seems to be an overkill for my setup.  I'll dump and reload and use
48MB bdb cachesize or possibly 32MB.  The number of threads uses a fair
amount of RAM, 8MB per thread if you use dbd backend, so reducing the
amount of threads helps as well.  I have not found the performance
difference between 16 (default) and 12.  I will further reduce this number
to 8 and see what happens.  I am still not sure about the cachesize
setting in the slapd.conf.  I started with cachesize of 100,000.
Reducing this number to 10,000 did not change the performance.  I am not
still not certain this parameter has any benefit if bdb is tuned?

Currently, sendmail is attaching to this ldap server and it is generating
20-50 queries per second.  Load on the machine is between .2 and 1.

Hope this helps.

-- 
Igor