Re: scaling openldap for performance

[Trevor Warren <trevorwarren@yahoo.com>]

Hello there,

 So with OpenLDAP 2.2.12 and the DB patches the System
Under Test no longer crashes/seg faults with a DB that
is un-recoverable???.

 At all times through your tests you have been able to
sustain without a *single* seg fault and didn't have
to restore to DB...right????.

 Could you care to throw some light on the System
Under Test(SUT) from the hardware perspective and
workload the SUT is being subjected to.

Trevor


--- RW <openldap@tauceti.net> wrote:
> Hi!
> 
> In the case someone is still interested in  this
> discussion: I'have done 
> my tests again
> now with OpenLDAP 2.2.12, BDB 4.2.52 (this time with
> the two patches 
> provided
> by Sleepycat) on the same machine with the same OS.
> Now everything works 
> fine.
> 
> Cheers,
> Robert
> 
> RW wrote:
> 
> > Hi!
> >
> > I think the discussion Dave mentioned was my
> question about "BDB 
> > locker" and
> > deadlocks occuring from time to time. Let me make
> one thing clear: A SAN
> > (Storage Area Network) normally consists of a SAN
> controller, a SAN 
> > switch
> > and a HDD array (available from different companys
> like EMC, HP, 
> > Hitachi Data
> > Storage,...) "behind" the SAN switch. It's like
> having directly 
> > connected a SCSI
> > HDD on a Adaptec SCSI controller. Do not confuse
> this with a NAS (Network
> > Attached Storage). We have a Compaq EVA SAN and
> beleave me this is fast
> > as hell ;-) especially the wait time's are almost
> not measurable and 
> > the throughput
> > is fantastic. If it's good for some big Oracle
> databases it should be 
> > good for
> > OpenLDAP ;-)
> >
> > I have done the same tests as Trevor and can
> comfirm his observations. 
> > I've loaded
> > OpenLDAP 2.2.11 with BDB 4.2.52 (WITHOUT the two
> patches available from
> > Sleepycat which solves some locking issues) on a
> Fujitsu Siemens RX300 (
> > 2 x 2.8 GHz, 2 GB RAM, Redhat ES 3) with about 1
> mio. real entries. If 
> > have
> > done some performance testing with Apache JMeter.
> I've also adjusted 
> > the parameters
> > in DB_CONFIG accordingly.
> >
> > Reading is simply really fast. But if you are
> doing a lot of modify's 
> > and/or add's on
> > our system if the server has really a lot to do
> I've also seen that 
> > OpenLDAP
> > crashes and database files could'nt be recoverd.
> Not even with the 
> > Sleepycat
> > tools. I was forced to recreate the database. This
> caused me sleepless 
> > nights.
> > But we have a replica so we can switch quickly if
> this happens. As far 
> > as I
> > remember the log output wasn't very helpfull. But
> I will try this on 
> > monday
> > again.
> >
> > This week I've compiled a new BDB 4.2.52 with the
> two patches from 
> > Sleepycat.
> > On monday I will do the performance test again.
> Maybe this patches 
> > will solve
> > the problem.
> >
> > Cheers,
> > Robert
> >
> >
> > Dave Lewney wrote:
> >
> >> Trevor Warren wrote:
> >>
> >>> Hey there Fellas,
> >>>
> >>>   Having tread through the archives for a long
> time i
> >>> have concluded with some load/performance tests.
> >>>
> >>>  Openldap scales *wonderfully* with respect to
> read
> >>> performance. Towards testing the same for write
> >>> performance i performed the following:
> >>>
> >>>
> >>>> With Zero think time we hit openldap with 20
> >>>
> >>>
> >>>
> >>> simultaenous users. The db has 1million records.
>  5
> >>> mins thro the test i have a seg fault. I try
> >>> restarting the same but the server refuses to
> re-read
> >>> the db unless i re-create the same. Find this
> very
> >>> strange. Can i avoid this In-consistency????
> >>>
> >>>
> >>>> Me brought this DB size to 100 Addressbook
> records
> >>>
> >>>
> >>>
> >>> and pounded the server with 20users and Zero
> think
> >>> time. Still server crashes.
> >>>
> >>>
> >>>> Brought the users to 10 users and 100 records.
> >>>
> >>>
> >>>
> >>> Bombarding the server with Zero think time. It
> now
> >>> survives and pulls through the test.
> >>>
> >>>  HW Arch is : HP BLade20p, 2 CPU, 1GB RAM, RHEL
> AS2.1,
> >>> RAID 0+1 on a SAN.
> >>>
> >>>  Am quite confused to be true.  Logging is
> disabled
> >>> cause we found its over head is simply too much
> and
> >>> hinders response times.
> >>>  Lemme know of any runtime/compile time options
> >>> towards boosting performance please. Thanks in
> >>> advance.
> >>>  
> >>> Trevor
> >>>
> >>
> >> If you are using the BDB backend then apparently
> there are problems 
> >> with using this on a SAN. I seem to remember this
> being discussed on 
> >> this list fairly recently.
> >>
> >> Dave
> >> -- 
> >> Dave Lewney
> >> Principal Systems Programmer, IT Services
> >> University of Sussex, Brighton BN1 9QJ. Tel:
> 01273 678354 Fax: 01273 
> >> 271956
> >>
> >>
> >>
> >
> >
> 


=====
( >-                                           -< )
/~\    ______________________________________   /~\
|  \) /    Scaling FLOSS in the Enterprise   \ (/ |
|_|_  \        trevorwarren@yahoo.com        / _|_|
       \____________________________________/


	
		
__________________________________
Do you Yahoo!?
Friends.  Fun.  Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/