RE: dissection of search latency

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Lawrence
> Greenfield

> A search with no indexing isn't a very interesting benchmark; this
> isn't what we should be optimizing for.

Unfortunately back-bdb's indexing support is still incomplete so doing it
any other way wouldn't make a very interesting benchmark either. Besides,
slowdowns revealed here will still be relatively slow in the indexed case.
But yes, it will also be necessary to analyze the timing for indexed
searches using some large number of non-sequential queries.
>
> Berkeley db with txns is fairly slow at iterations, though 0.5 seconds
> to iterate across 10000 entries seems pretty bad.  Is there any
> locking going on besides the internal locks in Berkeley db?  (On the
> plus side, the Berkeley db code will allow multiple iterators
> simultaneously.)

No, back-bdb does no locking of its own, everything is left up to Berkeley
db. Makes me wonder about the approach though; I don't see back-bdb being
viable as a general-purpose backend with such a high performance cost. Yes,
some people will sleep better at night knowing they can recover from
catastrophic disk failure, but those data-paranoid people also run redundant
hardware already, and don't need to be coddled by overprotective software.

> Also, Berkeley db does internal caching; no I/O goes on the second
> time around anyway.

True... However, it's probably still better to keep data in the slapd
internal format than in the disk format, especially for frequently
re-accessed entries.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support