[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Speed of slapadd for BDB on a Linux machine



--On Tuesday, November 07, 2006 5:04 PM +0100 Ralf Narozny <rnarozny@web.de> wrote:

Hello,

I'm trying to migrate about 19 million entries from OpenLDAP 2.0 to the
new 2.3. Conversion and such things are done, but inserting the data
takes days.

I hope someone can point me to some helpful direction, because 22
Entries/sec is not too good, especially not for real life use...

I'm using the following to insert the data
slapadd -q -v -c

My DB_CONFIG looks like this:

set_cachesize 1 524288000 1
set_lg_regionmax 262144
set_lg_bsize 2097152

It looks like from your stats that slapd has consumed all of the memory given to it, and now has to swap. You need to adjust your DB_CONFIG file so that there is more memory available to BDB, assuming of course you have enough RAM for that.

Since you have dual CPU's, you could set "tool-threads 2" in slapd.conf, as well.

Basically, if you want your slapadd to run quickly, you need to have enough RAM on the system, and the DB_CONFIG file configured, so that the DB can fit into RAM while being loaded. That's why my systems have 8GB and may expand to 16GB before too long as we continue to expand the data in the servers.

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html