[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: Questions...
> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Quanah Gibson-Mount
> --On Tuesday, January 27, 2004 2:35 PM -0500 Jong
> <jongchoi@OpenLDAP.org> wrote:
> > The difference between ldbm and bdb became significant when
> > indexing came into the picture.
> > So this may be a sign that we should improve the bdb indexing system
> > further.
>
> My data loads always find indexing to be the biggest
> bottleneck with BDB.
> Our systems that index multiple things went up two hours in
> indexing time
> between 2.1 & 2.2 with BDB (from 7 hours to 9 hours).
Well now that you mention it... The index generation logic is the same
between ldbm and bdb. The only difference is that ldbm uses Btrees for
everything by default, and back-bdb uses hashes for the indexes. When I
recompiled back-bdb to use Btrees the load time went down from 18m to 12m:
2.2 hdb (no txn logging)
691.310u 16.220s 12:26.53 94.7% 0+0k 0+0io 76127pf+0w
total 583102
-rw-r--r-- 2 hyc users 208 Jan 27 02:32 DB_CONFIG
-rw------- 1 hyc users 8 Jan 27 18:30 __db.001
-rw------- 1 hyc users 29724672 Jan 27 18:42 cn.bdb
-rw------- 1 hyc users 131985408 Jan 27 18:42 dn2id.bdb
-rw------- 1 hyc users 431702016 Jan 27 18:43 id2entry.bdb
-rw------- 1 hyc users 1343488 Jan 27 18:42 objectClass.bdb
Apparently growing a hash table on the fly is very expensive. The test008
seems to run about one second (10-11%) faster now as well.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
Symas: Premier OpenSource Development and Support