Re: Performance of MDB and BDB Please suggest?

[Howard Chu <hyc@symas.com>]

Chris Card wrote:
> 
> Hi all,
> 
> I am also doing some testing with mdb at the moment, and my initial
> testing
indicates that mdb is faster for reads but slower for writes than bdb.

This is generally true, and is documented in the papers I've published on the
subject.

> I am using openldap 2.4.32 on centos 6, on a 24 core box with 132 Gb RAM.

> My test directory has ~ 3 million entries, and I loaded it into mdb using
slapadd which took over 2 days (by comparison, the same load into bdb takes
2-3 hours).
> (as an aside, I initially tried using 2.4.31, but slapadd crashed after
having loaded about 90% of the data, and this was repeatable).

This is not normal. With slapadd -q MDB is faster than BDB assuming you're
using a decent filesystem and sensible mount options. JFS, EXT2, do better
than other filesystems in my tests. Very recent EXT4 may be better than EXT3
as well.

> On disk the directory takes up ~ 13 Gb for mdb and ~ 18Gb for bdb. Cache
> size for bdb is set to 63 Gb in DB_CONFIG. Directory size for mdb is set to
> 63 Gb

The relative sizes sound right.

> Adding 120000 entries from an ldif file using ldapadd took ~ 10 minutes
> for
mdb and ~ 2 minutes for bdb.
> Deleting 120000 entries using ldapdelete took ~ 10 minutes for mdb and ~ 3
minutes for bdb

Again, this is filesystem dependent, but basically right. If you're doing a
purely serial test, however, you're not seeing the true picture. With multiple
concurrent writers, BDB's throughput will degrade (due to transaction
deadlocks) while MDB's will remain constant. Again, this is documented in the
published papers on MDB.

MDB's write speed on my tests generally remains constant with a standard
deviation of essentially zero under all load conditions. BDB's write speed
will degrade under heavy load (to slower than MDB) and the standard deviation
widens as load increases.

> A search returning ~ 300000 DNs took ~ 6 seconds for mdb and for bdb it
> took
~ 6 minutes from a cold start of slapd and then ~ 35 seconds.

Sounds right. MDB's cold start overhead is measurable but essentially zero.

> Chris

-- 
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/