Re: bulk load speeds bdb vs ldbm

[Quanah Gibson-Mount <quanah@stanford.edu>]

--On Friday, April 22, 2005 9:43 AM -0400 Mark J Strawcutter 
<mjstraw@iup.edu> wrote:

> We run an OpenLDAP server to provide a "white pages" directory
> that is queried by email clients and a web look-up form.  Nothing
> complicated, complete reload of the database each night using
> an extract from out student/employee records system.
>
> ldbm has been working fine.  We can reload all 30k entries in
> a couple of minutes using slapadd.
>
> Common wisdom seems to be that bdb is better, so I'm investigating
> switching.  The first problem is that the load takes 2-3 hours :-(
> The slapd.conf params I've found that look like they might help
> have been set thusly:
>
> cachesize       100000
> idlcachesize    1000000
>
> Any suggestions on what I can do to bring bdb load times more in line
> with ldbm?  Or should I just stick with what's working (ie ldbm)?

I suggest you read over the many many posts, some quite recent, about 
properly configuring your DB_CONFIG file, which is a must with BDB. The two 
options you are currently examining are related to when the server is 
running, not when it is being loaded via slapadd.

See the OpenLDAP FAQ for more information.
<http://www.openldap.org/faq/data/cache/1166.html> is probably a good place 
to start, and reading,
<http://www.openldap.org/faq/data/cache/893.html> is of particular 
importance, and
<http://www.openldap.org/faq/data/cache/1072.html> will probably add some 
value.

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html

"These censorship operations against schools and libraries are stronger
than ever in the present religio-political climate. They often focus on
fantasy and sf books, which foster that deadly enemy to bigotry and blind
faith, the imagination." -- Ursula K. Le Guin