RE: Questions...

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Quanah Gibson-Mount

> --On Tuesday, January 27, 2004 2:35 PM -0500 Jong
> <jongchoi@OpenLDAP.org>
> wrote:

> > The difference between ldbm and bdb became significant when
> > indexing came into the picture.
> > So this may be a sign that we should improve the bdb indexing system
> > further.
>
> My data loads always find indexing to be the biggest
> bottleneck with BDB.
> Our systems that index multiple things went up two hours in
> indexing time
> between 2.1 & 2.2 with BDB (from 7 hours to 9 hours).  I was
> able to drop
> this to 2 hours (7 hours less!) by use the new shared memory
> key.  On our
> master (which indexes only 3 or 4 things), the load time takes
> approximately 25 minutes, on both 2.1 and 2.2.

With my recent changes in HEAD, slapindex time has improved quite a bit. I
have a 400MB database that takes ~52 minutes to slapadd, with 3 indices. It
used to take several hours (10 or more) to slapindex this database. It now
takes 1:06 hours, with IDL cache size explicitly set to 1 (effectively
disabled). With the IDL cache size at its new default (10000, given entry
cachesize of 1000) it takes only 11 minutes.

The use of the IDL cache should also have a significant improvement on
slapd's runtime indexing speed, not just slapindex.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support