[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: Slapd frontend performance issues

OK.  Rather than speculating on the frontend bottlenecks(if any), I'll setup
profiling next week to see where the time is spent. I'll then post the
results on this thread so we can further investigate.  Thanks.

-- Eric


-----Original Message-----
From: owner-openldap-devel@OpenLDAP.org
[mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Kurt D. Zeilenga
Sent: Thursday, April 12, 2001 10:31 PM
To: Eric T. Blue
Cc: openldap-devel@OpenLDAP.org
Subject: RE: Slapd frontend performance issues


At 03:41 PM 4/10/01 -0400, Eric T. Blue wrote:
>but I've been using the Netscape 4.1SDK.  Do you think there will be
>improvement by using the OpenLDAP libs?

I would suspect not.

>Also, do you have any suggestions
>as to what may be contributing to the front-end slowness?
>If there are
>suggestions to strip out extra "features", such as ACIs, I'd be willing to
>go with a bare-bones server for now just to see some better numbers.

ACIs don't cost anything if not enabled by an ACL.

Kurt


>-----Original Message-----
>From: owner-openldap-devel@OpenLDAP.org
>[mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Kurt D. Zeilenga
>Sent: Saturday, April 07, 2001 12:00 PM
>To: Eric T. Blue
>Cc: openldap-devel@OpenLDAP.org
>Subject: Re: Slapd frontend performance issues
>
>
>BTW, if you want to measure max. request thoughput, write
>a client which issues N search requests (using async API)
>upon the root DSE and then for each resultDone return,
>issue another request.  Set N to 100 or so.
>
>Kurt
>
>At 04:49 PM 4/6/01 -0400, Eric T. Blue wrote:
>>I've been working with Dmitry over the last couple months in trying to
fine
>>tune back-sql.  One avenue I'm looking at is implementing a backend
>>connection pool for the RDBMS, and possibly porting the ODBC code to
Oracle
>>specific OCI.  I've done a good bit of benchmarking so far with the ldbm
>>backend, as well as back-sql.  The other day I decided to start looking at
>>the performance of the OpenLDAP server as a whole, eliminating the
backend,
>>just to see what type of numbers I could expect.  Here are some numbers
>>benchmarked on a SUN E220 2/450Mhz procs w/ 2GB RAM:
>>
>>This is done via an in-house tool I wrote, but the numbers are almost
>>identical to those reported from Mindcraft's DirectoryMark.
>>non-threaded single client:
>>
>>BerkleyDB backend: 217 queries / second
>>BACK-SQL backend: 6 queries / second
>>
>>Now, I first tried to make a dummy backend myself which just defined the
>>initialize routine from backend.c.  In the init code, I assigned 0 to all
>>fcns except for the search fcn.  In the search fcn all I did was either
>>return an LDAP_SUCCESS message, or an LDAP_OTHER with some text to see
what
>>was going on.  The result was about 300 q / s.  Now, to give a more
>accurate
>>stress test, I ran 20 clients in parallel which did a bind, search, and
>>unbind for every operation.  Combining the results of all 20 clients
>>increases the number to show about 570 q /s with no backend.  I also tried
>>returning the LDAP_SUCCESS directly from search.c in servers/slapd.  This
>>number seems low to me, and I was wondering if there is anything I can do
>to
>>increase performance?  For some reason, I made an assumption that the
>>"slowness" in throughput was due to disk or protocol IO for the backend
>>processing... maybe not.  My ultimate goal, which may be unrealistic at
>this
>>point, is to achieve a MAX number of 1,000 q / s while using OCI.  What
I'd
>>like to do is establish a connection pool of x number of persistent OCI
>>connections to Oracle on DB startup and assign each LDAP handle to a
>>particular OCI handle for the connection.  I'd appreciate any insight
>>anybody can offer.  You guys have been very helpful so far.  Thanks!
>>
>>-----Original Message-----
>>From: mitya@samovaar.dot.ru [mailto:mitya@samovaar.dot.ru]On Behalf Of
>>Dmitry Kovalev
>>Sent: Friday, April 06, 2001 5:55 AM
>>To: Eric T. Blue
>>Subject: Re: back-sql / oc question
>>
>>
>>Hello Eric!
>>
>>"Eric T. Blue" wrote:
>>
>>> Dmitry,
>>>         I haven't had a chance to profile it yet, but I may have some
>>dissapointing
>>> news.  Up till now, I've assumed the a great deal of processing is spent
>>on
>>> the backend (regardless if it is ldbm, or back-sql).  So, I decided to
>>make
>>> dummy backend that really does nothing... just inits function pointers
to
>>0.
>>> I've also changed search.c in the servers/slapd directory to give an
>>> LDAP_SUCCESS message and return instantly.  If I do this and then give a
>>> stress test to the server (ie. 10 clients connecting simultaneously with
>>> 1,000 searches each) I only can achieve 300 queries per second.  If this
>>is
>>> the case, then obviously I can never expect to achieve a number greater
>>than
>>> this(or even 25% of this assuming that an OCI backend is put in place).
>>> This number is very surprising to me, and I can't believe that the core
>>> OpenLDAP code is this sluggish.. what do you think?
>>>
>>
>>Well, this is too strange (all it have to do is choose a working thread,
>>parse the
>>request, and send OK), but after all - I have no knowledge of slapd
>>frontend - I just
>>plugged in my backend, thats all...
>>
>>
>>There are a number of tunings that can be done - worker thread pool size
>>etc., of
>>which I don't know much...
>>
>>We should contact somebody from the real core - because my core membership
>>is limited
>>to back-sql support - I don't have much time even for this nowadays :(
>>
>>Namely, Kurt D. Zeilenga appears to be the chief architect - we could
>>forward this to
>>him, with questions regarding slapd frontend...
>>
>>WBW, Dmitry