[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: performance

Howard Chu wrote: 
Ultimately after several more repeated runs, AD peaked at 4800 auths/sec (a
bit faster than the 4400 we saw before, apparently it took a while for their
caches to get fully primed). We then removed AD and installed ADAM. The import
went a bit faster this time, thankfully:

time ldifde.exe -i -h -f examp3.ldif -s localhost -q 8
152.62u 99.32s 27:33.84 15.2%

Using 8 threads, 27-1/2 minutes. The peak authentication rate we got was just
under 5500 auths/sec using 52 client threads. At that load a single auth took
an average of 9.5 milliseconds, up from 1.5ms at 4 client threads. (slapd's
average latency is only 0.5ms at 8 clients, 2.2ms at 52 clients.)

A slapadd -q for back-hdb on 1M entries took only 5 minutes:
time ../servers/slapd/slapd -Ta -f slapd.conf.slam -q -l ~hyc/ldif/example.ldif.1mil

real    5m2.669s
user    3m59.224s
sys     1m55.482s

(Using BDB 4.6.21 and libtcmalloc)

An ldapadd (with back-bdb) of those 1M entries took 19 minutes:
time ../clients/tools/ldapmodify -a -x -D dc=example,dc=com -w secret -f ~hyc/ldif/example.ldif.1mil

real    19m3.044s
user    1m12.435s
sys     0m38.401s

So, even restricted to a single thread, we still import entries faster than AD or ADAM with 8 threads. Online or offline, they don't even come close. (Which should be no surprise...)

It'll be even more fun when we come back and run the tests on a 5 million
entry DB. It's pretty clear from looking at their memory usage that they won't
be able to cache all of that in the 16GB of RAM on this box, while OpenLDAP
will. As such, their throughput rate will just reflect the speed of our disks,
while slapd will still be running at full RAM speed.


The time to slapadd 5 million entries for back-hdb was 29 minutes:
time ../servers/slapd/slapd -Ta -f slapd.conf.5M -l ~hyc/5Mssha.ldif -q

real    29m15.828s
user    20m22.110s
sys     6m56.433s

For ADAM it was about 3 hours:

time ldifde.exe -i -h -f 5M.ldif -s localhost -q 8
743.57u 475.56s 2:59:44.78 11.3%

The authrate for back-hdb was again 25,700/sec with SSHA passwords, pretty much the same as for 1 million entries. The authrate for ADAM was 2359/sec.

It's also interesting to note the database and process sizes. The 5M ldif was about 2.9GB; the hdb database was about 6.4GB. The ADAM DB was about 22GB.

The slapd process size was about 8GB. The ADAM process size was about 14GB. We could easily handle a 10 million entry database on this machine and still be running at full cache speed; ADAM is already disk limited here.

This weekend we'll try loading 5M into AD. I figure we'll start up the import at the end of the day today and come back to look at it again in a couple of days...
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/