[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: back-bdb cachesize and DB_CONFIG set_cachesize

Thank you for all that information Howard.

Due to problems with the make test test008-concurrency hanging (machine is dual 800 Mhz PIII on Via Apollo Pro 133A) I upgraded my RedHat 9 with 2.1.22 and dependency RPMs from RedHat/Rawhide so it should allow me to tweak the BDB settings.  I'm pretty sure I was having a linking issue where by the proper version of BDB wasn't being picked up by the OpenLDAP compile (performance when I tested some of our data wasn't stellar compared to the RPM install either).  I'm pretty new to Linux though I've used Solaris as a development environment for years.  I'm a software engineer rather than sysadmin so much of the compile dependencies for Linux/Unix are not things I'm very familiar with.  If anyone has suggestions about how to fix my RedHat 9 build of OpenLDAP then I'm all ears.

In the mean time I'll play with the BDB caches a little more and see what I get.  Many thanks!

Jamey

-----Original Message-----
From: Howard Chu [mailto:hyc@symas.com]
Sent: Friday, October 10, 2003 1:52 AM
To: James Courtney; 'Open LDAP Software (E-mail)'
Subject: RE: back-bdb cachesize and DB_CONFIG set_cachesize


> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of James Courtney
 
> Hey all,
> 	How is the cachesize configuration option used for OpenLDAP? 

It's not used by "OpenLDAP" - it is used by particular backends in the slapd server that is part of OpenLDAP. There are other "cache sizes" throughout the package, and the meaning depends a great deal on the specific context you're talking about.

> My current user database has 20,000 users each 
> with additional sub-dn's for probably about 50,000 actual 
> dn's.  Does this mean that a cachesize of 50,000 would store 
> all of my dn's in memory?  Is the whole object stored or just 
> specific attributes and/or indexes?

The whole entry is stored. That's why the manual page refers to it as an entry cache. If your database contains 50,000 objects or less, then a cachesize of 50000 will store all the objects in memory, yes.

> Is cachesize relative to 
> a particular connection (session) or index or just simply the 
> number of objects which will be cached by the OpenLDAP.

cachesize is a configuration keyword specified in an individual slapd backend database. It applies to that database, not to a session, and not to the server as a whole. A single server may have multiple database instances, and each database may have its own entry cache.

> What causes an object to be removed from cache?

The cache follows a Least-Recently-Used replacement strategy. Old objects are removed from the cache when the cache hits its configured size and new objects are requested.

> If an object is queried for using a different attribute and different index 
> is the one in memory searched or is a new one loaded from disk first?

Searching on an indexed attribute uses the indexing logic, which is separate from the entry cache. In back-bdb there is a separate cache (the IDL cache) that is used for caching index data. Obviously the point of an in-memory cache is to reduce the need to access the disk. The indexer produces a candidate list of entry IDs corresponding to entries that may possibly fulfill the search. The entries matching these IDs are fetched and tested against the search parameters. The fetch will use the entry cache. Entries are only loaded from the disk if they aren't already present in the cache.

> Also, how do I set the Berkeley DB set_cachesize parameter as 
> I can see no DB_CONFIG file for my DB 4.1?  I did a RedHat 
> RPM install so I don't know what the BDB home directory is.  
> I suspect that OpenLDAP passes it to the BDB subsystem using 
> the DB_ENV->open call but what is it set to?

A RedHat RPM install would give you OpenLDAP 2.0, which does not use BDB environments, so the question is moot. If you do in fact have OpenLDAP 2.1 then the BDB home directory is wherever you set the "directory" to in slapd.conf.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support