RE: FW: slapd threads on master pegs multiple CPU Sun running Solaris 8 (ITS#2302)

["Howard Chu" <hyc@highlandsun.com>]

Thinking about this more, my previous post didn't make much sense... In a
regular (non-transactional) read operation, BDB releases locks as soon as the
read finishes. I think this is the ideal mode of operation for reads.
Wrapping reads in a transaction means locks will accumulate until the end of
the transaction, which increases the probability of a particular read causing
another write to block. If we just break the search operation into many
little read transactions, this is no different than the locking behavior of
non-transactional reads. So, I still don't see any advantage to be gained
here, and I don't see that the BDB library itself needs any changes.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

> -----Original Message-----
> From: Kurt D. Zeilenga [mailto:Kurt@OpenLDAP.org]

> One possible approach is not to wrap the whole search operation
> in a single transactions, but use many.  One for baseObject
> finding, one for candidate determination, and one for each
> candidate.  However, this would likely cost way more than 10%
> (with logging).
>
> Kurt
>
>
> At 04:11 PM 2/14/2003, Howard Chu wrote:
> >I guess it's worth exploring. It might be nice if BDB
> provided a locker
> >entity for this purpose - i.e., like a transaction, all
> locks are held by one
> >locker, but without any of the logging associated with a
> real transaction. I
> >haven't convinced myself yet that it really buys us much.
> >
> >In the normal case, you slow down the search by 10% and get
> no other benefit.
> >
> >In the case of multiple concurrent operations, you actually decrease
> >concurrency because locks obtained inside a transaction are
> held to the end
> >of the transaction, no matter if you explicitly release them or not.
> >
> >In the case of a search spawned while processing another
> transaction, you are
> >creating nested transactions, which decreases concurrency
> even fruther
> >because all of the locks in each nested transaction are held
> until the parent
> >is released. If you don't nest the transactions, then you're just as
> >vulnerable to deadlock as before.
> >
> >The deadlock problem seems to arise from the fact that we
> can either use
> >transactions, in which case all locks are tied to that one
> transaction (this
> >is good because it prevents us from deadlocking ourself, bad
> because it keeps
> >locks longer than they're needed) or we have every
> individual operation
> >getting/releasing its own locks on the fly (bad because we
> can deadlock
> >ourself easily). What would be ideal is something in the
> middle - passing a
> >locker ID to every BDB function, so that it gets used for
> all the locks, and
> >they are all released at the exact moment we wish.
> >
> >  -- Howard Chu
> >  Chief Architect, Symas Corp.       Director, Highland Sun
> >  http://www.symas.com               http://highlandsun.com/hyc
> >  Symas: Premier OpenSource Development and Support
> >
> >-----Original Message-----
> >From: owner-openldap-bugs@OpenLDAP.org
> >[mailto:owner-openldap-bugs@OpenLDAP.org]On Behalf Of
> jongchoi@us.ibm.com
> >
> >I think we should restart discussing on the
> transaction-protected searches.
> >The overhead of transactional searches does not seem to be
> prohibitively
> >significant.
> >(I remember around 10% when I experimented with it.)
> >One concern is the semantics of the aborted LDAP search.
> >1. abort the whole search ?
> >    In this case, it would be the client's responsibility to
> resend the
> >search request.
> >2. abort a single db->get() of the candidate loop ?
> >   We should have one transaction per loop iteration.
> >   The overhead of frequent transaction begin and commit has to be
> >measured.
> >- Jong
> >
> >BTW, Joseph, what is the version number of BerkeleyDB ? Is it 3.x ?
> >
> >------------------------
> >Jong Hyuk Choi
> >IBM Thomas J. Watson Research Center - Enterprise Linux Group
> >P. O. Box 218, Yorktown Heights, NY 10598
> >email: jongchoi@us.ibm.com
> >(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979
>