Re: Slapd frontend performance issues

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

BTW, if you want to measure max. request thoughput, write
a client which issues N search requests (using async API)
upon the root DSE and then for each resultDone return,
issue another request.  Set N to 100 or so.

Kurt

At 04:49 PM 4/6/01 -0400, Eric T. Blue wrote:
>I've been working with Dmitry over the last couple months in trying to fine
>tune back-sql.  One avenue I'm looking at is implementing a backend
>connection pool for the RDBMS, and possibly porting the ODBC code to Oracle
>specific OCI.  I've done a good bit of benchmarking so far with the ldbm
>backend, as well as back-sql.  The other day I decided to start looking at
>the performance of the OpenLDAP server as a whole, eliminating the backend,
>just to see what type of numbers I could expect.  Here are some numbers
>benchmarked on a SUN E220 2/450Mhz procs w/ 2GB RAM:
>
>This is done via an in-house tool I wrote, but the numbers are almost
>identical to those reported from Mindcraft's DirectoryMark.
>non-threaded single client:
>
>BerkleyDB backend: 217 queries / second
>BACK-SQL backend: 6 queries / second
>
>Now, I first tried to make a dummy backend myself which just defined the
>initialize routine from backend.c.  In the init code, I assigned 0 to all
>fcns except for the search fcn.  In the search fcn all I did was either
>return an LDAP_SUCCESS message, or an LDAP_OTHER with some text to see what
>was going on.  The result was about 300 q / s.  Now, to give a more accurate
>stress test, I ran 20 clients in parallel which did a bind, search, and
>unbind for every operation.  Combining the results of all 20 clients
>increases the number to show about 570 q /s with no backend.  I also tried
>returning the LDAP_SUCCESS directly from search.c in servers/slapd.  This
>number seems low to me, and I was wondering if there is anything I can do to
>increase performance?  For some reason, I made an assumption that the
>"slowness" in throughput was due to disk or protocol IO for the backend
>processing... maybe not.  My ultimate goal, which may be unrealistic at this
>point, is to achieve a MAX number of 1,000 q / s while using OCI.  What I'd
>like to do is establish a connection pool of x number of persistent OCI
>connections to Oracle on DB startup and assign each LDAP handle to a
>particular OCI handle for the connection.  I'd appreciate any insight
>anybody can offer.  You guys have been very helpful so far.  Thanks!
>
>-----Original Message-----
>From: mitya@samovaar.dot.ru [mailto:mitya@samovaar.dot.ru]On Behalf Of
>Dmitry Kovalev
>Sent: Friday, April 06, 2001 5:55 AM
>To: Eric T. Blue
>Subject: Re: back-sql / oc question
>
>
>Hello Eric!
>
>"Eric T. Blue" wrote:
>
>> Dmitry,
>>         I haven't had a chance to profile it yet, but I may have some
>dissapointing
>> news.  Up till now, I've assumed the a great deal of processing is spent
>on
>> the backend (regardless if it is ldbm, or back-sql).  So, I decided to
>make
>> dummy backend that really does nothing... just inits function pointers to
>0.
>> I've also changed search.c in the servers/slapd directory to give an
>> LDAP_SUCCESS message and return instantly.  If I do this and then give a
>> stress test to the server (ie. 10 clients connecting simultaneously with
>> 1,000 searches each) I only can achieve 300 queries per second.  If this
>is
>> the case, then obviously I can never expect to achieve a number greater
>than
>> this(or even 25% of this assuming that an OCI backend is put in place).
>> This number is very surprising to me, and I can't believe that the core
>> OpenLDAP code is this sluggish.. what do you think?
>>
>
>Well, this is too strange (all it have to do is choose a working thread,
>parse the
>request, and send OK), but after all - I have no knowledge of slapd
>frontend - I just
>plugged in my backend, thats all...
>
>
>There are a number of tunings that can be done - worker thread pool size
>etc., of
>which I don't know much...
>
>We should contact somebody from the real core - because my core membership
>is limited
>to back-sql support - I don't have much time even for this nowadays :(
>
>Namely, Kurt D. Zeilenga appears to be the chief architect - we could
>forward this to
>him, with questions regarding slapd frontend...
>
>WBW, Dmitry