[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#4702) out-of-memory on huge DB ?
--On Friday, October 13, 2006 1:57 PM +0000 Paolo.Rossi.con@h3g.it wrote:
>> No. As I already mentioned, there is no reason for slapcat to use
>> large amounts of RAM, it should use whatever size your BDB cache is
>> configured for and that's about it. It's possible there is a memory
>> leak, but since no such leak appears under Linux that would point
>> to either the Solaris C library or some other library specific to
>> your platform.
>
> Today I've finished a cross test on a suse 10.0 linux
>
> Athlon XP 2200+ 768MB RAM + 1GB swap
>
>
>
> loaded huge db with same config, shared mem and set_cachesize 0
> 384000000 1
>
> slapcat -f slapd.conf
So you understand that without the -l flag to slapcat, you are writing to
stdout instead of a file? Is it possible that is what is causing slapcat
to crash?
On my linux 2.6 system (64 bit), with a 2.5 million entry db, which is:
ldap00:/var/lib/ldap/slamd/64-2.5mil-db42# du -h *.bdb
84M cn.bdb
618M dn2id.bdb
85M employeeNumber.bdb
3.2G id2entry.bdb
3.7M objectClass.bdb
85M uid.bdb
DB_CONFIG is:
set_cachesize 0 384000000 1
set_lg_regionmax 262144
set_lg_bsize 2097152
set_lg_dir /var/log/bdb/slamd/64-2.5mil
set_lk_max_locks 3000
#
# Automatically remove log files that are no longer needed.
set_flags DB_LOG_AUTOREMOVE
#set_flags DB_DIRECT_DB
#
# Setting set_tas_spins reduces resource contention from multiple clients
on systems with multiple CPU's.
set_tas_spins 1
Using slapcat -l ldif_file, I then ran a while (1) loop to check the OSS
and RSS sizes of slapcat. It reached the following size fairly quickly:
RSS VSZ
1159740 1191916
and then never grew past that. This means the resident memory maxed out at
1.1 GB, and the virtual memory maxed out at 1.1GB as well.
So, that does mean one thing -- It definitely outgrew the 366MB of BDB
cache defined. On the other hand, once it reached its max size, it stayed
steady there until completion was reached.
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html