Re: Very slow search speed

["Paweena Charoentham" <paweenac@ksc.net>]

Dear Igor,

With 16 simultaneous clients,

- ldapsearch tool consumes 3.716 seconds/request.

- My external program consumes 5.798 seconds/request.

            - ldap_simple_bind_s: 3.953 seconds

            - ldap_search_ext_s "(&(username=USERNAME)(realm=REALM))":
0.9384 seconds

            - ldap_search_ext_s “(products=PRODUCT)”: 0.889 seconds



After search on the net, I found some paper said that some of the connect
latency come from Nagle algorithm. To disable it, I compile openldap-2.1.12
with –DTCP_NODELAY (Is this the right way to disable it?) and I test again,
due to there are some change on 2.0.27 and 2.1.12, I maintain almost
configuration of the old version and discard only some that are not
compatible.



loglevel             4



threads             50

concurrency       40

cachesize          1000000

#dbcachesize     1000000000



# Indices to maintain

index   objectClass         eq

index   products             eq

index   uid                     eq

index   username           eq

index   realm                 eq

index   mail                    eq

index   domain               eq



The respond time from both ldapsearch and my extAuth increase about 1 second
for every request. (Is this come from Berkeley DB? How can I tune it?) The
number of threads while running test is still 9 and not increase.



Paweena Charoentham


----- Original Message -----
From: "Igor Brezac" <igor@ipass.net>
To: "Paweena Charoentham" <paweenac@ksc.net>
Cc: "openldap-software" <openldap-software@OpenLDAP.org>
Sent: Friday, March 14, 2003 9:37 AM
Subject: Re: Very slow search speed


>
> On Fri, 14 Mar 2003, Paweena Charoentham wrote:
>
> > Dear Igor,
> >
> > In authentication part, I do not use FreeRadius authentication
> > configuration. Because I need some special features so I have to write
my
> > own code and call it as an external program.
> >
> >
> > To bypass authentication request to my external program, I put this
> > configuration on FreeRadius’ users file.
> >
> >
> >
> > DEFAULT Auth-Type := Accept
> >
> >         Exec-Program-Wait =
> > "/usr/local/freeradius/radiusExternalProgram/bin/extAuth"
> >
> >
> >
> > I bind ldap with manager’s dn and password. The search filters are
> >
> > 1. "(&(username=USERNAME)(realm=REALM))"
> >
> > and
> >
> > 2. "(products=PRODUCT)"
> >
>
> How long does "ldapsearch '(&(username=USERNAME)(realm=REALM))'" take?
>
> -Igor
>
> >
> >
> > On ldap, all users refer by ‘uid’ (= combination of ‘username’ + ‘realm
’)
> > are under ‘products’ branch.
> >
> > Paweena Charoentham
> >
> > ----- Original Message -----
> > From: "Igor Brezac" <igor@ipass.net>
> > To: <openldap-software@OpenLDAP.org>
> > Sent: Thursday, March 13, 2003 9:04 PM
> > Subject: Re: Very slow search speed
> >
> > > Please provide your freeradius ldap related configuration.  In
particular
> > > show filter and groupmembership_filter parameters.
> > >
> > > -Igor
> > >
> > > On Thu, 13 Mar 2003, Paweena Charoentham wrote:
> > >
> > > > I want to use OpenLDAP with FreeRadius but after some tests I have a
big
> > problem with performance.
> > > >
> > > > For 1 client with continuously search using ldapsearch command the
> > throughput rate is only 3-4 requests/second. (There are about 600,000
> > entries on ldap) I also test by put only 10,000 entries on ldap but the
> > throughput rate is not different much.
> > > >
> > > > When I increase the number of simultaneous client continuously send
> > requests to ldap. The throughput rate sharply decreases. With 16
> > simultaneous clients, the throughput rate is 0.2 request/second. Over 16
> > clients, OpenLDAP will saturate.
> > > >
> > > > I also write my own client program using OpenLDAP library to measure
> > which part drop the performance down. My client open connection to ldap,
do
> > 2 searches and then close the connection. After run it, I found that for
> > each search request, 66% of time spends on ldap_simple_bind_s function.
Each
> > ldap_search_ext_s takes 16% and 2% for others.
> > > >
> > > > On this test I use OpenLDAP-2.0.27 on RedHat 7.3 machine, Intel Xeon
2
> > GHz dual CPU with 2 GB RAM, Adaptec SCSI host adapter (aic7899) on
RAID5,
> > BROADCOM NetXtreme BCM5702X Gigabit Ethernet Adapter (on board). Number
of
> > the data in the ldap is about 600,000 entries.
> > > >
> > > > The following is some tuning I made on slapd.conf
> > > >
> > > > threads             32
> > > > concurrency       27
> > > > loglevel             4
> > > > # Indices to maintain
> > > > index   objectClass         eq
> > > > index   products             eq
> > > > index   uid                     eq
> > > > index   username,realm  eq
> > > > index   mail                    eq
> > > > index   domain               eq
> > > >
> > > > cachesize                      1000000
> > > > dbcachesize                   1000000000
> > > >
> > > > (I also do this test on OpenLDAP-2.1.12 and the throughput rate is
as
> > same as 2.0.27)
> > > >
> > > > While the test is running, the ldap server load is only about 2 and
> > there is some physical memory left. But I wonder why the number of
threads
> > is always being 9? Why does it not increase to share works? Am I do
> > something wrong? Or what is the right way to tune OpenLDAP to gain an
> > acceptable performance with this enormous data? Anyone help me please…
> > > >
> > > > Paweena Charoentham
> > > >
> > >
> > > --
> > > Igor
>
> --
> Igor