RE: HEADS UP: new back-bdb caching code

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> The cache also allows shortcutting the dn2id_children
> routine: if the cache's
> child AVL tree is non-NULL then the node definitely has
> children. (If NULL,
> you still need to check the database. Perhaps we should add a
> flag to the
> cache structure that says "this cached node definitely has no
> children" ...)

The test results are extremely artificial, but using this shortcut appears to
reduce deadlock occurrences by 25% in test008. Overall performance impact
isn't too visible since there are plenty of other lock conflicts going on.

> This is not the same as the BDB_HIER hierarchical database
> code, but it is
> certainly a necessary step in that direction.

(BDB_HIER is still broken in the current CVS, needs to be adapted for the new
cache.)

> The cache uses nested transactions to coordinate database
> updates with cache
> updates. Changes are only applied to the cache after all of the nested
> database operations have committed successfully. This greatly
> simplifies the
> cache state management.

The change to only acquire Read locks at the beginning of an operation
instead of RMW locks appears to have also improved concurrency. Again,
minimizing the duration of Write locks...

> At present my tests show it to be about the same speed as the
> previous code.
> Hopefully it isn't any slower...

Probably need to revisit the sequence of events in updating the databases.
The BDB documentation advises updating the most heavily contentious items
last, to minimize the amount of time those resources are locked. In test008
the most heavy conflict is in the dn2id index. We should probably reorder the
operations to always update id2entry first; it's not clear to me whether
dn2id will always be the worst offender in the presence of extensive
attribute indexing but we should probably update it last.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support