strange performance problem Update

[xoror@infuse.org]

Hi all,

The last 2 weeks i've tried various things to try to solve the strange
behaviour that i'm seeing. I'll try to give a short summary and the
results of them.

1. tried suggestions of the mailing list:
- disk problem -> can't be i got 5GB free hdd left
- Cache / memory prob -> i got 200 MB of free ram. (how does one set cache
parameter for bdb4? like the cache parameters for ldbm.)
- locking problem
db_* utilities provided some usefull information. i've made a log of
locking overviews during the testruns. But i don't see an explanation
(yet) for the weird behaviour. 
- concurrency problems with my test programms.
 i've double checked the code and i'm pretty confident that there are no
concurrency problem. There's only 1 shared resource and access to it is
controlled with/by a mutex.

2. I've upgraded from bdb 4.1.24 to 4.1.25
this somehow helps a bit. With 4.1.25 i'm getting the weird behaviour
after 10 testruns instead of 4/5. Somehow the new bdb seems to 'fix' this
a bit (partially).

3. tried to get ldbm working with openldap 2.1.x
This failed, i don't know why exactly but my geuss is that it's a library
problem. (i got bdb 1, bdb 3 and bdb4 installed)

4. I installed openldap 2.0.27 on a new machine to test it with ldbm. The
results where very surprising. 2.0.27 with ldbm doesn't show the weird
behaviour. It also only used 20% cpu power max for 32 clients. i've run
the tests like 80 times now, and i'm pretty confident that this version
doesn't have the problem described below. The performance is quite good.
the avarage testrun time was about 6 seconds for 1000 task.

I'm suspection that there's somehow something wrong in either 2.1.x or in
bdb4 implementation. I'm doing this research as part of my master thesis.
I'm really want to get 2.1.x working correctly since this one should have
transactional capabilities (wich are important to the application i had
in mind). any hints on possible problem are welcome. i'll be glad to look
in to it. 

for now i'm digging in the bdb documentations (and changelog to see what
they changed from .24 to .25 that can be related to my problem).

thank you for your time,
Cuong

On Sun, 23 Mar 2003, Peter Marschall wrote:

> Hi,
> 
> have you only tred it with back-dbd od with back-ldbm too ?
> Just to get an idea if it is more related to the core of slapd or to a
> specific backend.
> 
> Peter
> 
> 
> 
> On Sunday 23 March 2003 14:47, Cuong bui wrote:
> > Hello all,
> >
> > I'm doing some research on openldap to see if it's suitable for a certain
> > kind of application. This application requires more write operations from
> > the ldap server then an 'average' usage of ldap.
> >
> > For this purpose i've written an multithreaded application to simulate
> > simultaneous access to the ldap server. This application creates an
> > variable number of threads that simulate an client. There's a central queue
> > of task that have to be performed. The queue has 3 kind of task (for now).
> >
> > 1. filter task. Simple search on a certain key
> > 2. modify task. modification of a 'record'.
> > 3. authentication task. (simple bind and unbind, simple authentication for
> > now)
> >
> > The task queue is built in such a way that the type of tasks are mixed with
> > eachother.
> >
> > i've generated a test dataset of 50.000 records and the .bdb files have
> > been copied to an other location (ofcourse  when the server is offline) in
> > order to restore data (instead of regenerating the test dataset, it can now
> > be copied back).
> >
> > The testruns were very promising. for instance with 1000 tasks (300 filter,
> > 600 modification, 100 authentication) and 32 threads only took 9 seconds to
> > complete. However performing this same test a few times gives me a rather
> > surprise effect. the first 4 runs all took about 9-13 seconds to complete.
> > The 5th run and up takes up to 5 mins (!) or more.  Simple searches with
> > ldapsearch takes up a few mins (after 4 testruns). (before that they only
> > took a fraction of a second).
> >
> > during the first 4 testruns the openldapserver used 98% cpu processing
> > time. (seen in top) After the initial 4 runs, slapd only consumes 40%-75%
> > cpu processing time. somethimes it's even dropping to a few %.
> >
> > The only way to get slapd working 'normal' again, is to shutdown the server
> > and restore the dataset. I have defined an index on  the field that i'm
> > filtering on. (without it searches will take a very long time to complete).
> >
> > The test server consist of a pentium 4 2,53 ghz running gentoo linux with
> > kernel 2.4.20 and ultra dma (enabled) hdd   with 512 MB ram. Openldap
> > 2.1.16 was running on this machine. (but also tested with 2.1.12 and 2.1.15
> > they all gave this result) BDB was chosen as backend for this test.)
> >
> > The testclient consist of an pentium 4 1,8ghz with gentoo linux running
> > kernel 2.4.19. The 2 systems were connected on a 100 mbit (switched)
> > network. All the software were compiled with g++ 3.2.1
> >
> > Has anyone an explanation for this behaviour ? Is this a known problem ?
> > All hints are welcome.
> >
> >
> > Thank in advance (and for your time)
> > Cuong
> >
> > btw: are there any documentation on implementations details of openldap ?
> > if yes, where can i find them :) Other documentation that might explain
> > what i'm seeing is also (very) welcome.
> 
> -- 
> Peter Marschall
> eMail: peter@adpm.de
> 
>