[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: restricting slapd memory consumption
Quanah Gibson-Mount schrieb:
--On Wednesday, April 02, 2008 10:41 AM +0200 Ralf Narozny
<rnarozny@web.de> wrote:
Yep, as I wrote in my initial mail, we are using 2.3.32 (for testing so
far).
The current release is 2.3.41. Why are you using such an old release?
There've been many known problems fixed since that release, including
security issues, and possibly some memory leaks.
We are using that version, because it is a project that is not highly
priorized and the package was compiled long a ago. But I just
initialized an upgrade. Memory leaks are for sure a show stopper.
What version of BDB are you using? Have you patched it with all the
relevant patches?
We are using BDB 4.4.20. With as far as I know no patches. Any
recommended version or neccessary patches?
And I wrote that we are using BDB which is configured to use 4GB of
shared mem. The only problem I have is that with 1000000 entries
configured as entry cache, slapd uses 11GB out of 16 GB of RAM after the
insert with ldapadd. Which makes it use 7GB for entry cache (and
whatever
else).
What's the contents of your DB_CONFIG file? What's the size du -c -h
*.bdb in your database directory? What is your idlcachesize setting?
How many file descriptors do you allocate to slapd?
DB_CONFIG:
set_flags DB_LOG_AUTOREMOVE
set_cachesize 4 0 2
set_lg_max 524288000
set_lg_regionmax 512000
set_lg_bsize 268435456
du -c -h *.bdb:
792M cid.bdb
5,7G cn.bdb
4,1G dn2id.bdb
36M folderName.bdb
15G id2entry.bdb
948K locked.bdb
1,9M objectClass.bdb
idlcachesize: 0, because we only use it for customer logins, so there
are no searches besides the ones for the customer id
If you mean the number of file descriptors ready to be used, it is 4096
currently, but might be raised, if neccessary.
Since you are using ldapadd to populate the database (rather than
slapadd), the entry cache is getting immediately allocated. As I
described in a previous email writeup I did on performant tuning of
slapd, the DB_CONFIG setting is the most important (to hold the size
of your entire DB), followed by the entry cache size.
Yep, that is my problem, we will surely need a lot of entries in cache,
because, we got more than 2 million customers logging in per day.
Can you tell me the subject of the mail writeup so I can find it in the
list archive?
The entry cache really only needs to be the size of your active set of
entries, rather than all entries. For example, at a previous
position, even though we had about 500,000 entries in our database,
our entry cache was only 20,000. And it was very performant.
We got 23 million entries of which more than 2 million are active
entries...we can not really reduce the size of our database. Maybe a few
thousands, but not millions ;-)
Our entries have (in LDIF of course) an average size of below 200 Bytes.
So taking 6GB out of the 7GB used as size of the entry cache, it would
mean that each entry consumes about 6K of RAM. Is that correct?
If so, is there any documentation on how to configure the slapd for a
larger amount of entries like ours?
How many entries do you have?