Re: OpenLDAP performance vs. PostgreSQL

[apm@mutex.dk]

On Thu, Mar 16, 2006 at 12:31:19PM -0500, John Madden wrote:
> > I've only got the 20000 simple entries. I've narrowed it down to only one
> > integer attribute now and increased the cache to 100Mb, but it's still much
> > slower than DBD::Pg.
> 
> ...And what version of openldap are you running again?

2.2.23 (The version in Debian Sarge). Debian hasn't even gotten to 2.3.x in unstable yet.
 
> To give you some idea of what you'll be able to do with the proper 
> configuration: our directory hear is about 300,000 objects, each holding a 
> couple dozen attributes.  With a loglevel of 256, we regularly serve 5000 
> requests/sec on a dual Xeon 3.0.  I'm sure I could do better with some 
> tweaking, too, but it hasn't been necessary.  (And we use a 3GB bdb cache, 
> btw, although that's obviously not all consumed -- but with 8GB RAM...)

Yes... it seems very tempting to pursue. But right now I tempted to just give up and store the user config in PostgreSQL.

The problem is really simple. I have a web-application (mod_perl)
which have ~100000 users and I'll have to load their individual config
once per request. I expect heavy load, but very few writes.

Doing a simpe test (below) of fast that can be done with PostgreSQL
shows reasonable performance, but if OpenLDAP really is faster, I
would really prefer to do that since it'll be easier for me to
integrate to other services later. (I have an WebDAV-server I would
like to authenticate for with LDAP too).


So... how do I get OpenLDAP to do the equivalent of this ... only
faster (for the query part)?:

================================================================
#!/usr/bin/perl 

use DBI;
use DBD::Pg;


my $dbh =
DBI->connect("dbi:Pg:dbname=sql_test;host=localhost","apm","password") |
| die $DBI::errstr;

# Drop tables

my $rv = $dbh->do("DROP table objects") or warn $dbh->errstr;

# Create tables

$rv = $dbh->do("CREATE table objects (id serial, pif integer,primary key(id))") 
or die $dbh->errstr;

# Insert data

my $t1 = time;

my $sth = $dbh->prepare("INSERT INTO objects (pif) VALUES (?)");

for (my $i = 1; $i<=20000; $i++) {
    $sth->execute($i);
}

my $t2 = time;
print "INSERT: ",$t2-$t1,"\n";
$t1=$t2;

# Look up data on index

for (my $i = 1; $i<=20000; $i++) {
    my @row_array = $dbh->selectrow_array("SELECT * FROM objects WHERE id=$i");
}

my $t2 = time;
print "LOOKUP: ",$t2-$t1,"\n";