RE: Slow to add 1 million items

[Chris Card <ctcard@hotmail.com>]

________________________________

> Hello all, 
> 
> I've been Google'ing around and searching the archives, but I haven't 
> quite been able to find an answer, so I wanted to ask the list. 
> 
> I've been experimenting with OpenLDAP adds to see how quickly we can 
> get data inserted into the DB. 
> 
> I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 
> 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using 
> lmdb, both with relatively similar results. 
> 
> The short version is: to insert 1 million records, it's taking about 8 
> hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long 
> time to me. 
> 
> The insert method is to use a single big ldiff file like this: 
> 
> ### snip ### 
> dn: cn=newtest0,ou=customer,dc=test,dc=com 
> objectclass: inetOrgPerson 
> userPassword: 90sac90sudasdjcao0sdjtest 
> telephoneNumber: 61400000000 
> sn: none 
> 
> dn: cn=newtest1,ou=customer,dc=test,dc=com 
> objectclass: inetOrgPerson 
> userPassword: 90sac90sudasdjcao0sdjtest 
> telephoneNumber: 61400000000 
> sn: none 
> ### snip ### 
> 
> Then add it with ldapadd: 
> ldapadd -x -D cn=admin,dc=test,dc=com -w test -f ./test 
> 
> <snipped>
> 
> Any ideas or things I could try? 
> 
slapadd will be much faster

Chris