[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: openldap versions and silent exit

Sorry, just in a rush to make the Cavalia show in Berkeley with my girlfriend and hit reply:)

I've been running db_stat a lot to look at the cache hit rate which is now 100%.

I've got no locks granted after waiting and the thing has been up for 18 hours or so.  My output is below.  Why don't I see a "Total cache size."?

I'm not sure I have the luxury of storing DB logs on a separate disk as our servers use a HW mirrored RAID 1 with two 36 GB 10k RPM SCSI disks which are quite fast.  Also, I believe that the DB logs are only written for updates of which we honestly have very few so I can't imagine that hurting performance much.  Is my understanding here correct?  I realize that in an ideal world good DB practice is to store the logs on a separate physical drive (just like Oracle or SQL Server) but as we don't have the disks for that now but we have several servers with the data and regular ldif snapshots I'm feeling like we're okay.

Can you please elaborate on what idlcachesize actually does?

I also have been assuming that setting cachesize > than the number of entries would just allow us to grow to 50k entries from our 20k without someone having to reconfig but would have no adverse affect in the mean time.  Is this logic fair?

Thanks so much for your help!

Jamey


[root@DC-UMAIL01 openldap-data]# /apps/db/4.2.52/bin/db_stat -m
        Total cache size.
1       Number of caches.
40MB 8KB        Pool individual cache size.
0       Requested pages mapped into the process' address space.
9483036 Requested pages found in the cache (100%).
606     Requested pages not found in the cache.
0       Pages created in the cache.
606     Pages read into the cache.
97      Pages written from the cache to the backing file.
0       Clean pages forced from the cache.
0       Dirty pages forced from the cache.
0       Dirty pages written by trickle-sync thread.
1659    Current total page count.
1586    Current clean page count.
73      Current dirty page count.
4099    Number of hash buckets used for page location.
9484247 Total number of times hash chains searched for a page.
3       The longest hash chain searched for a page.
11M     Total number of hash buckets examined for page location.
19M     The number of hash bucket locks granted without waiting.
0       The number of hash bucket locks granted after waiting.
0       The maximum number of times any hash bucket lock was waited for.
1279    The number of region locks granted without waiting.
0       The number of region locks granted after waiting.
615     The number of page allocations.
0       The number of hash buckets examined during allocations
0       The max number of hash buckets examined for an allocation
0       The number of pages examined during allocations
0       The max number of pages examined for an allocation



-----Original Message-----
From: Quanah Gibson-Mount [mailto:quanah@stanford.edu]
Sent: Tuesday, August 10, 2004 6:44 PM
To: James Courtney
Cc: openldap-software@openldap.org
Subject: RE: openldap versions and silent exit




--On Tuesday, August 10, 2004 9:00 PM -0400 James Courtney 
<Jcourtney@inphonic.com> wrote:

> Hey Quanah,
>
> DB_CONFIG:
># Set cachesize to 32MB
> set_cachesize 0 33554432 0
>
> slapd.conf:
> cachesize 100000 (~20k entries actually in the DB)
>
> idlcachesize?  I don't have this anywhere
>
> Backend is BDB 4.2.52.

Hi,

Please don't email me directly, or at least CC the list. :)

Your DB_CONFIG might be too small.  There are a number of ways to check.

I suggest running "db_stat -m" in your DB directory and look for things 
like a high number of locks granted after waiting

Here's output from my test master:

ldap-test0:/db# db_stat -m
2GB     Total cache size.
1       Number of caches.
2GB     Pool individual cache size.
0       Requested pages mapped into the process' address space.
119M    Requested pages found in the cache (100%).
16      Requested pages not found in the cache.
112213  Pages created in the cache.
16      Pages read into the cache.
313145  Pages written from the cache to the backing file.
0       Clean pages forced from the cache.
0       Dirty pages forced from the cache.
0       Dirty pages written by trickle-sync thread.
112229  Current total page count.
112229  Current clean page count.
0       Current dirty page count.
262147  Number of hash buckets used for page location.
119M    Total number of times hash chains searched for a page.
2       The longest hash chain searched for a page.
134M    Total number of hash buckets examined for page location.
605M    The number of hash bucket locks granted without waiting.
418773  The number of hash bucket locks granted after waiting.
29      The maximum number of times any hash bucket lock was waited for.
380048  The number of region locks granted without waiting.
109     The number of region locks granted after waiting.
112269  The number of page allocations.
0       The number of hash buckets examined during allocations
0       The max number of hash buckets examined for an allocation
0       The number of pages examined during allocations
0       The max number of pages examined for an allocation



As you can see, the number of region locks after waiting (109) is quite low 
compared to the number granted without waiting (380,000)

Also, the number of hash bucket locks granted after waiting (418,000) is 
quite low compared to the number granted without waiting (605 Million)

There is no need to set cachesize higher than your total number of entries.

The backend you are using is determined in slapd.conf in the "database XXX" 
parameter.  That backend, if it is ldbm or bdb, is then overlayed on top of 
BDB 4.2.52.


Also, if that is all there is to your DB_CONFIG file, you may also want to 
set some other parameters like:

set_lg_regionmax 262144
set_lg_bsize 2097152
set_lg_dir <YOUR LOG DIRECTORY>

Note that storing the BDB logs on a separate disk or spindle dramatically 
increases the performance of OpenLDAP.

idlcachesize might be a 2.2 only option, I'm not sure, I abandoned 2.1 a 
while ago.

--Quanah


--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html