FW: slapd threads on master pegs multiple CPU Sun running Solaris 8 (ITS#2302)

["Howard Chu" <hyc@highlandsun.com>]

I guess it's worth exploring. It might be nice if BDB provided a locker
entity for this purpose - i.e., like a transaction, all locks are held by one
locker, but without any of the logging associated with a real transaction. I
haven't convinced myself yet that it really buys us much.

In the normal case, you slow down the search by 10% and get no other benefit.

In the case of multiple concurrent operations, you actually decrease
concurrency because locks obtained inside a transaction are held to the end
of the transaction, no matter if you explicitly release them or not.

In the case of a search spawned while processing another transaction, you are
creating nested transactions, which decreases concurrency even fruther
because all of the locks in each nested transaction are held until the parent
is released. If you don't nest the transactions, then you're just as
vulnerable to deadlock as before.

The deadlock problem seems to arise from the fact that we can either use
transactions, in which case all locks are tied to that one transaction (this
is good because it prevents us from deadlocking ourself, bad because it keeps
locks longer than they're needed) or we have every individual operation
getting/releasing its own locks on the fly (bad because we can deadlock
ourself easily). What would be ideal is something in the middle - passing a
locker ID to every BDB function, so that it gets used for all the locks, and
they are all released at the exact moment we wish.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

-----Original Message-----
From: owner-openldap-bugs@OpenLDAP.org
[mailto:owner-openldap-bugs@OpenLDAP.org]On Behalf Of jongchoi@us.ibm.com

I think we should restart discussing on the transaction-protected searches.
The overhead of transactional searches does not seem to be prohibitively
significant.
(I remember around 10% when I experimented with it.)
One concern is the semantics of the aborted LDAP search.
1. abort the whole search ?
    In this case, it would be the client's responsibility to resend the
search request.
2. abort a single db->get() of the candidate loop ?
   We should have one transaction per loop iteration.
   The overhead of frequent transaction begin and commit has to be
measured.
- Jong

BTW, Joseph, what is the version number of BerkeleyDB ? Is it 3.x ?

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979