OpenLDAP Developers' Day - Presentation

This presentation introduces a performance driven design effort for back-bdb, a transactional backend of OpenLDAP. Evolved from the back-ldbm design, back-bdb offers transactional semantics with enhanced reliability and concurrency by directly and fully utilizing the Berkeley DB transactional data store API. The back-bdb transaction support ensures the ACID property of directory operations and the transaction logging enables directory data recovery from transient errors and catastrophic system failures. Locking granularity is refined to a page level and a deadlock between competing threads is resolved by aborting and restarting a victim transaction so that multiple threads and applications can access the directory database concurrently. Back-bdb stores directory entries to the database in a binary format minimizing the number of memory allocations / deallocations. In addition, back-bdb simplified the management of IDL (ID List) of dn2id and indexing to reduce the inode style IDL management overhead of back-ldbm.

This presentation describes the design of two different caches to improve the back-bdb performance : the IDL stack slab cache and the IDL cache. Through a detailed performance evaluation and system-level profiling on Linux, the IDL stack allocation / deallocation and the IDL database fetch was identified as main performance bottlenecks. The IDL stack slab cache provides preallocated slabs for the IDL stacks which store IDLs from component search filters to construct combined search candidate lists. The IDL cache stores IDLs from dn2id and indexing databases, using the database keys as cache indices. The cache entry is invalidated upon insertion and deletion of an ID in the IDL, and will be re-fetched and re-cached at the next fetch of the IDL. Both caches were integrated into back-bdb without major refactoring of the exisitng code. Combined benefits of both caches are 31% improvement in the throughput and latency of back-bdb search for an industry directory benchmark suite. With these caches enabled, back-bdb can service high performance directory searches with transaction support, enhanced reliability and concurrency. This presentation will also discuss further performance improvements such as BER cache design for transfer contents.

This presentation is available in the following formats:

• OpenOffice
• PDF
• PowerPoint