Re: slapadd filching memory

["Jong" <jongchoi@OpenLDAP.org>]

> Openldap 2.1.23
> BDB 4.1.25
> RH9
>
> I recently added a very large amount of data to a fresh installation:
> about 200K entries with significant text and lots of interrelationships.
> The ldif file itself was over 11 Mb. I used slapadd.
>
> I monitored memory during the process. I saw free physical memory
> descend from 880 Mb to under 10 while disk cache grew from 110 Mb to 830
> and disk buffers grew from 10 Mb to 60. Then came the traditional dance
> of swapping memory around for each operation and moving at the pace of a
> sedated snail. On a final run it amused me to let the process keep
> going, and I'm happy to report that (after many hours) all the data did
> eventually load successfully. Searches on my fat new store perform
> admirably.
>
> The problem is that the memory is not released, neither during the
> slapadd (which would've helped), nor after the process is completed,
> (successfully or not). Starting and stopping slapd does nothing. I have
> to restart the machine to get the memory back. I fiddled with DB cache
> values (4 Mb in a single block seemed to work best); that helped
> efficiency somewhat but memory keeps disappearing in the same manner.

4Mb DB cache size of the second case is too small a value wrt the ldif size.
What is the DB cache value of the first case ?
I suspect the first thrashing is at the OS layer while the second is at the
BDB layer.
This means that the DB cache size was set either too high or too low.
Try some values in between...
- Jong

> My question: is this normal, something that could come from a
> misconfiguration, or ITS territory warranting further investigation?
> Could this be related to the other BDB discussion?
>
> It may just be all my doing, but I would appreciate any advice on where
> to start looking for a fix. TIA.
>
> Jon Roberts
> www.mentata.com
>
>
>
>