Re: Slow to add 1 million items

Thanks, Chris.

Yeah, I hear you on that, but sorry, to be more specific, I was running this test to get an idea of performance for regular LDAP use, and slapadd is a purely offline solution.

It would be helpful for a restore, of course, but not equivalent to when our application will be adding/modifying records (which is what I'm really trying to simulate).

Running the same test of inserting 1M rows into postgres with the same type of data record on the same machine goes about 3x faster, which just doesn't sound right, since LDAP should be way faster than Postgres, right?

Cheers,

Andrew

Andrew Eross
CTO
Locatrix Communications
Office: +61 7 3123 1469

Mobile: +55 37 9858 9815
eross@locatrix.com

On Tue, Feb 4, 2014 at 8:25 AM, Chris Card <ctcard@hotmail.com> wrote:

________________________________

> Hello all,
>
> I've been Google'ing around and searching the archives, but I haven't
> quite been able to find an answer, so I wanted to ask the list.
>
> I've been experimenting with OpenLDAP adds to see how quickly we can
> get data inserted into the DB.
>
> I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP
> 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using
> lmdb, both with relatively similar results.
>
> The short version is: to insert 1 million records, it's taking about 8
> hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long
> time to me.
>
> The insert method is to use a single big ldiff file like this:
>
> ### snip ###
> dn: cn=newtest0,ou=customer,dc=test,dc=com
> objectclass: inetOrgPerson
> userPassword: 90sac90sudasdjcao0sdjtest
> telephoneNumber: 61400000000
> sn: none
>
> dn: cn=newtest1,ou=customer,dc=test,dc=com
> objectclass: inetOrgPerson
> userPassword: 90sac90sudasdjcao0sdjtest
> telephoneNumber: 61400000000
> sn: none
> ### snip ###
>
> Then add it with ldapadd:
> ldapadd -x -D cn=admin,dc=test,dc=com -w test -f ./test
>

> <snipped>

>
> Any ideas or things I could try?
>

slapadd will be much faster

Chris