Re: (ITS#5249) OpenLDAP memory leak

[hyc@symas.com]

sumith.narayanan@gmail.com wrote:
> Full_Name: Sumith Narayanan
> Version: 2.3.27
> OS: MacOSX 10.4
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (17.216.21.198)

There are two known memory leaks related to syncrepl and one related to ACL 
sets in 2.3.x releases since 2.3.27. Your 2.3.27 is quite old, the current 2.3 
is 2.3.39.

There are no known leaks related to the loglevel though. That sounds more like 
a memory leak in your platform's C library, in the syslog() function. You 
would need to use a malloc debugger to verify that.

> My openLDAP DB is divided into 3 physical DBs, each of 4 GB , 12 GB and 26 GB
> sizes , 42 GB total.
> 
> We were running with ldbm backend till 6-7 months back when the total size of
> the DB was below 30 GB. It started crashing once in a while as the size started
> increasing.
> 
> We then changed the backend to  Berkeley DB (Version:4.4.20) and the openLDAP
> version 2.3.27. It is running in a 8 GB memory machine with 32 bit processing
> and receives updates through an custom interface which binds to ldap and do the
> update / inserts and deletes.
> 
> The rate at which the updates/inserts/deletes come is around 1-2 K / hr.
> 
> Now when I start the slapd process in loglevel 256 or 512 , it starts crashing
> with in a few hours running out of memory. The maximum memory that 32 bit
> processor can use is around 2 GB. It crashes after that. I was trying to tune
> the DB_CONFIG  :
> 
> DB_CONFIG settings for each DB :
> 
> For 26 GB DB.
> set_cachesize   0       42428800        0
> # Transaction Log settings
> set_lg_regionmax        1048576
> set_lg_max              20485760
> set_lg_bsize            2097152
> 
> For 12GB DB
> set_cachesize   0       15485760        0
> set_lg_regionmax        1048576
> set_lg_max              20485760
> set_lg_bsize            2097152
> 
> for 4 GB DB
> set_cachesize   0       10485760        0
> set_lg_regionmax        1048576
> set_lg_max              10485760
> set_lg_bsize            2097152
> 
> 
> Slapd.conf settings :
> 
> loglevel     16
> gentlehup          on
> idletimeout        300
> sizelimit          1000
> timelimit          300
> password-hash     {SSHA}
> allow bind_v2
> threads           32
> 
> These parameters are set same for all the three DBs :
> 
> dbcachesize     100000
> cachesize        100000
> 
> There is a replica also which gets updates from the master.
> 
> Now when I change the loglevel to 32 or 16 , the process lives longer ,
> sometimes for a week or for more than a month.... However , as soon as I switch
> back to old loglevel , it starts crashing. This is happening in both master and
> slave.
> 
> Could you please advice whether there is a fix for this in the current version
> or is it getting fixed in the coming version OR is it something that I can fix
> by configuring. Logs are important for us as we need to know where the requests
> are coming from. Also , the DB is accessed 250 K times in a day approximately.
> 
> Please let me know if I need to furnish any more details.
> 
> Thanks, Sumith.
> 
> 
> 


-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/