[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: [openldap 2.1.12 + BerkeleyDb 4.1.24] db_checkpoint pb





--On Tuesday, November 25, 2003 6:14 PM +0100 Helene.Lastennet@alcatel.fr wrote:

Hi,
I made some tests with the 2 solutions:
1. checkpoint directive in cron job (my reference about this solution:
http://www.openldap.org/lists/openldap-bugs/200306/msg00096.html)
2. checkpoint directive in slapd.conf "checkpoint 10000 60" according to
your advice

My test was the same for the 2 solutions : mass-provisioning of 1000 new
subscriptions (1 subscription = 4 ldap entry in bdb1 backend + 3 ldap
entry in bdb2 backend => cf. slapd.conf for backends definition)

Here's my results:
solution 1:
time to execute the provisioning = 6mn43s
only 2 running slapd with %CPU = 1-2% in average
db_stat with option -e, -m, -t,  -c, -l  in the 2 backend seems to be
correct

solution2:
time to execute the provisioning = 20mn28s
8 running slapd with %CPU = 20% in average (with for one of them a peak =
32%CPU)
db_stat with options -m, -t,  -c in the 2 backend seems to be correct
db_stat with option -l gives a bad result: "1355 The number of region
locks granted after waiting." for bdb1 backend
and "3411 The number of region locks granted after waiting." for bdb2
backend

Cf. in attachment the result of db_stat command for solution n°2 (only
bdb1 backend)
Cf. in attachment too the DB_CONFIG file of the bdb1 backend for example

So my question is: How doing to obtain best results with solution n°2
(with tuning BerkeleyDB ??)
the solution n°1 being disadvised ?

Helene,

How are you doing your mass provisioning? ldapadd? If that is the case, these results don't surprise me. ldapadd is very inefficient for doing bulk dataloads. The checkpointing will definitely slow things down in that case as well, since slapd will be checkpointing the data periodically during the load. slapadd is a much more efficient bulk-loading solution. The other option is to turn off checkpointing while bulk loading.

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITSS/TSS/Computing Systems
ITSS/TSS/Infrastructure Operations
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html