RE: Slapd frontend performance issues

["Eric T. Blue" <openldap@star-cs.com>]

Kurt,
	Thanks.  I'll give that a try.  I'm not sure if it will make a difference,
but I've been using the Netscape 4.1SDK.  Do you think there will be
improvement by using the OpenLDAP libs?  Also, do you have any suggestions
as to what may be contributing to the front-end slowness?  If there are
suggestions to strip out extra "features", such as ACIs, I'd be willing to
go with a bare-bones server for now just to see some better numbers.

-----Original Message-----
From: owner-openldap-devel@OpenLDAP.org
[mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Kurt D. Zeilenga
Sent: Saturday, April 07, 2001 12:00 PM
To: Eric T. Blue
Cc: openldap-devel@OpenLDAP.org
Subject: Re: Slapd frontend performance issues


BTW, if you want to measure max. request thoughput, write
a client which issues N search requests (using async API)
upon the root DSE and then for each resultDone return,
issue another request.  Set N to 100 or so.

Kurt

At 04:49 PM 4/6/01 -0400, Eric T. Blue wrote:
>I've been working with Dmitry over the last couple months in trying to fine
>tune back-sql.  One avenue I'm looking at is implementing a backend
>connection pool for the RDBMS, and possibly porting the ODBC code to Oracle
>specific OCI.  I've done a good bit of benchmarking so far with the ldbm
>backend, as well as back-sql.  The other day I decided to start looking at
>the performance of the OpenLDAP server as a whole, eliminating the backend,
>just to see what type of numbers I could expect.  Here are some numbers
>benchmarked on a SUN E220 2/450Mhz procs w/ 2GB RAM:
>
>This is done via an in-house tool I wrote, but the numbers are almost
>identical to those reported from Mindcraft's DirectoryMark.
>non-threaded single client:
>
>BerkleyDB backend: 217 queries / second
>BACK-SQL backend: 6 queries / second
>
>Now, I first tried to make a dummy backend myself which just defined the
>initialize routine from backend.c.  In the init code, I assigned 0 to all
>fcns except for the search fcn.  In the search fcn all I did was either
>return an LDAP_SUCCESS message, or an LDAP_OTHER with some text to see what
>was going on.  The result was about 300 q / s.  Now, to give a more
accurate
>stress test, I ran 20 clients in parallel which did a bind, search, and
>unbind for every operation.  Combining the results of all 20 clients
>increases the number to show about 570 q /s with no backend.  I also tried
>returning the LDAP_SUCCESS directly from search.c in servers/slapd.  This
>number seems low to me, and I was wondering if there is anything I can do
to
>increase performance?  For some reason, I made an assumption that the
>"slowness" in throughput was due to disk or protocol IO for the backend
>processing... maybe not.  My ultimate goal, which may be unrealistic at
this
>point, is to achieve a MAX number of 1,000 q / s while using OCI.  What I'd
>like to do is establish a connection pool of x number of persistent OCI
>connections to Oracle on DB startup and assign each LDAP handle to a
>particular OCI handle for the connection.  I'd appreciate any insight
>anybody can offer.  You guys have been very helpful so far.  Thanks!
>
>-----Original Message-----
>From: mitya@samovaar.dot.ru [mailto:mitya@samovaar.dot.ru]On Behalf Of
>Dmitry Kovalev
>Sent: Friday, April 06, 2001 5:55 AM
>To: Eric T. Blue
>Subject: Re: back-sql / oc question
>
>
>Hello Eric!
>
>"Eric T. Blue" wrote:
>
>> Dmitry,
>>         I haven't had a chance to profile it yet, but I may have some
>dissapointing
>> news.  Up till now, I've assumed the a great deal of processing is spent
>on
>> the backend (regardless if it is ldbm, or back-sql).  So, I decided to
>make
>> dummy backend that really does nothing... just inits function pointers to
>0.
>> I've also changed search.c in the servers/slapd directory to give an
>> LDAP_SUCCESS message and return instantly.  If I do this and then give a
>> stress test to the server (ie. 10 clients connecting simultaneously with
>> 1,000 searches each) I only can achieve 300 queries per second.  If this
>is
>> the case, then obviously I can never expect to achieve a number greater
>than
>> this(or even 25% of this assuming that an OCI backend is put in place).
>> This number is very surprising to me, and I can't believe that the core
>> OpenLDAP code is this sluggish.. what do you think?
>>
>
>Well, this is too strange (all it have to do is choose a working thread,
>parse the
>request, and send OK), but after all - I have no knowledge of slapd
>frontend - I just
>plugged in my backend, thats all...
>
>
>There are a number of tunings that can be done - worker thread pool size
>etc., of
>which I don't know much...
>
>We should contact somebody from the real core - because my core membership
>is limited
>to back-sql support - I don't have much time even for this nowadays :(
>
>Namely, Kurt D. Zeilenga appears to be the chief architect - we could
>forward this to
>him, with questions regarding slapd frontend...
>
>WBW, Dmitry