RE: Slapd frontend performance issues

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 03:41 PM 4/10/01 -0400, Eric T. Blue wrote:
>but I've been using the Netscape 4.1SDK.  Do you think there will be
>improvement by using the OpenLDAP libs?

I would suspect not.

>Also, do you have any suggestions
>as to what may be contributing to the front-end slowness?
>If there are
>suggestions to strip out extra "features", such as ACIs, I'd be willing to
>go with a bare-bones server for now just to see some better numbers.

ACIs don't cost anything if not enabled by an ACL.

Kurt


>-----Original Message-----
>From: owner-openldap-devel@OpenLDAP.org
>[mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Kurt D. Zeilenga
>Sent: Saturday, April 07, 2001 12:00 PM
>To: Eric T. Blue
>Cc: openldap-devel@OpenLDAP.org
>Subject: Re: Slapd frontend performance issues
>
>
>BTW, if you want to measure max. request thoughput, write
>a client which issues N search requests (using async API)
>upon the root DSE and then for each resultDone return,
>issue another request.  Set N to 100 or so.
>
>Kurt
>
>At 04:49 PM 4/6/01 -0400, Eric T. Blue wrote:
>>I've been working with Dmitry over the last couple months in trying to fine
>>tune back-sql.  One avenue I'm looking at is implementing a backend
>>connection pool for the RDBMS, and possibly porting the ODBC code to Oracle
>>specific OCI.  I've done a good bit of benchmarking so far with the ldbm
>>backend, as well as back-sql.  The other day I decided to start looking at
>>the performance of the OpenLDAP server as a whole, eliminating the backend,
>>just to see what type of numbers I could expect.  Here are some numbers
>>benchmarked on a SUN E220 2/450Mhz procs w/ 2GB RAM:
>>
>>This is done via an in-house tool I wrote, but the numbers are almost
>>identical to those reported from Mindcraft's DirectoryMark.
>>non-threaded single client:
>>
>>BerkleyDB backend: 217 queries / second
>>BACK-SQL backend: 6 queries / second
>>
>>Now, I first tried to make a dummy backend myself which just defined the
>>initialize routine from backend.c.  In the init code, I assigned 0 to all
>>fcns except for the search fcn.  In the search fcn all I did was either
>>return an LDAP_SUCCESS message, or an LDAP_OTHER with some text to see what
>>was going on.  The result was about 300 q / s.  Now, to give a more
>accurate
>>stress test, I ran 20 clients in parallel which did a bind, search, and
>>unbind for every operation.  Combining the results of all 20 clients
>>increases the number to show about 570 q /s with no backend.  I also tried
>>returning the LDAP_SUCCESS directly from search.c in servers/slapd.  This
>>number seems low to me, and I was wondering if there is anything I can do
>to
>>increase performance?  For some reason, I made an assumption that the
>>"slowness" in throughput was due to disk or protocol IO for the backend
>>processing... maybe not.  My ultimate goal, which may be unrealistic at
>this
>>point, is to achieve a MAX number of 1,000 q / s while using OCI.  What I'd
>>like to do is establish a connection pool of x number of persistent OCI
>>connections to Oracle on DB startup and assign each LDAP handle to a
>>particular OCI handle for the connection.  I'd appreciate any insight
>>anybody can offer.  You guys have been very helpful so far.  Thanks!
>>
>>-----Original Message-----
>>From: mitya@samovaar.dot.ru [mailto:mitya@samovaar.dot.ru]On Behalf Of
>>Dmitry Kovalev
>>Sent: Friday, April 06, 2001 5:55 AM
>>To: Eric T. Blue
>>Subject: Re: back-sql / oc question
>>
>>
>>Hello Eric!
>>
>>"Eric T. Blue" wrote:
>>
>>> Dmitry,
>>>         I haven't had a chance to profile it yet, but I may have some
>>dissapointing
>>> news.  Up till now, I've assumed the a great deal of processing is spent
>>on
>>> the backend (regardless if it is ldbm, or back-sql).  So, I decided to
>>make
>>> dummy backend that really does nothing... just inits function pointers to
>>0.
>>> I've also changed search.c in the servers/slapd directory to give an
>>> LDAP_SUCCESS message and return instantly.  If I do this and then give a
>>> stress test to the server (ie. 10 clients connecting simultaneously with
>>> 1,000 searches each) I only can achieve 300 queries per second.  If this
>>is
>>> the case, then obviously I can never expect to achieve a number greater
>>than
>>> this(or even 25% of this assuming that an OCI backend is put in place).
>>> This number is very surprising to me, and I can't believe that the core
>>> OpenLDAP code is this sluggish.. what do you think?
>>>
>>
>>Well, this is too strange (all it have to do is choose a working thread,
>>parse the
>>request, and send OK), but after all - I have no knowledge of slapd
>>frontend - I just
>>plugged in my backend, thats all...
>>
>>
>>There are a number of tunings that can be done - worker thread pool size
>>etc., of
>>which I don't know much...
>>
>>We should contact somebody from the real core - because my core membership
>>is limited
>>to back-sql support - I don't have much time even for this nowadays :(
>>
>>Namely, Kurt D. Zeilenga appears to be the chief architect - we could
>>forward this to
>>him, with questions regarding slapd frontend...
>>
>>WBW, Dmitry