[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: Memory Usage Spikes



--On Thursday, March 18, 2010 3:47 PM -0400 "Beuerlein, Edward" <Edward_Beuerlein@cable.comcast.com> wrote:

Hi Quanah,
Thanks for replying!  Please let us know what other info you might need
to help us figure out these large increases in memory usage.

Here's the info you requested:

I did a dump of the database and then ran this on it(please let me know
if that isn't what you wanted):

# cat auth01.031610.ldif |grep dn |wc -l
60753

So you have 60,753 entries. Just to be sure, I would have used grep ^dn: to be exact. ;)

set_cachesize 0 268435456 1

You have 256MB allocated to BDB

69M     cn.bdb
26M     dn2id.bdb
3.7M    entryCSN.bdb
2.2M    entryUUID.bdb
160K    gidNumber.bdb
2.1G    id2entry.bdb
840K    ipHostNumber.bdb
44K     memberNisNetgroup.bdb
304K    memberUid.bdb
3.1M    nisNetgroupTriple.bdb
2.9M    objectClass.bdb
8.0K    ou.bdb
8.0K    sudoUser.bdb
3.0M    uid.bdb
364K    uidNumber.bdb
792M    uniqueMember.bdb
2.9G    total

Your BDB database is 2.9GB.

olcDbCacheSize 1000
olcDbCacheFree 1
olcDbDNcacheSize 0
olcDBIDLcacheSize 1000

You're only allowing the first 1000 entries to be cached in OpenLDAP, and your IdlCache is quite small as well. dncachesize is fine (unlimited).

While none of this explains your memory spikes, your server is definitely poorly tuned. It may be that your massive discrepancy between BDB cache allocation and actual BDB size may be causing the problem (in looking at the bdb error messages in your log).

I would highly advise the following changes.

For DB_CONFIG

set_cachesize 4 0 1

This will increase the BDB cache to 4GB (which can hold your 2.9GB DB easily)

For cn=config:

olcDbCacheSize 80000
olcDbCacheFree 1000
olcDBIDLcacheSize 240000

This will allow all entries to be cached in OpenLDAP, free up a reasonable number if you exceed the cache, and allow a decent IDL size.

I forgot to ask which slapd backend you are using (hdb or bdb), and which version of BDB you are using (and if it is fully patched) which is also useful information.

You may also wish to read over <http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0>, I think you guys have some experience with Zimbra... ;)

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration