[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: DB_CONFIG

Turbo Fredriksson wrote:

Quoting Quanah Gibson-Mount <quanah@stanford.edu>:



Second, '1736KB'. Where did that number come from!?


4*434.



Doh!


Just to clearify (with my information).

----- s n i p -----
[papadoc.root]# db4.2_stat -h /var/lib/ldap -d /var/lib/ldap/dn2id.bdb 53162 Btree magic number.
9 Btree version number.
Flags: duplicates, little-endian
2 Minimum keys per-page.
4096 Underlying database page size.
2 Number of levels in the tree.
583 Number of unique keys in the tree.
1329 Number of data items in the tree.
1 Number of tree internal pages.
3618 Number of bytes free in tree internal pages (12% ff).
18 Number of tree leaf pages.
34744 Number of bytes free in tree leaf pages (53% ff).
1 Number of tree duplicate pages.
2650 Number of bytes free in tree duplicate pages (35% ff).
0 Number of tree overflow pages.
0 Number of bytes free in tree overflow pages (0% ff).
0 Number of pages on the free list.
[papadoc.root]# db4.2_stat -h /var/lib/ldap -d /var/lib/ldap/id2entry.bdb 53162 Btree magic number.
9 Btree version number.
Flags: little-endian
2 Minimum keys per-page.
16384 Underlying database page size.
2 Number of levels in the tree.
272 Number of unique keys in the tree.
272 Number of data items in the tree.
1 Number of tree internal pages.
15876 Number of bytes free in tree internal pages (3% ff).
27 Number of tree leaf pages.
72442 Number of bytes free in tree leaf pages (84% ff).
0 Number of tree duplicate pages.
0 Number of bytes free in tree duplicate pages (0% ff).
1 Number of tree overflow pages.
12019 Number of bytes free in tree overflow pages (27% ff).
0 Number of pages on the free list.
----- s n i p -----

From this, I asume that my calculation would look like this: 

File: /var/lib/ldap/dn2id.bdb
 4096  Underlying database page size.
 1     Number of tree internal pages.
 18    Number of tree leaf pages.
 => ((1+1)*4096) => 40960

+ 

File: /var/lib/ldap/id2entry.bdb
 16384 Underlying database page size.
 1     Number of tree internal pages.
 27    Number of tree leaf pages.
 => ((1+1)*16384) => 32768


Resulting in the value 73728 (73Mb). Correct values used?


73728 bytes. You really should pay more attention to units when performing simple arithmetic.

Considering that my id2entry.bdb file is only 491520 bytes
and dn2id.bdb is 86016 bytes, I find it odd that I have to
to use a HUMONGOUSLY bigger cache than [whoever wrote FAQ
entry #1075]... Or did I missread '73728'?



You misread. And yes, I wrote the text of that FAQ article. Since I also wrote the code in back-bdb that is being described, you can be sure the description was accurate at the time of writing.

Followup question 2: It said on the page that one should
take into account any indexes...

The calculation example looks like this:

The objectClass index for my example database is 5.9MB and uses 3 hash
buckets and 656 duplicate pages. So ( 3 + 656 ) * 4KB / 2 =~ 1.3MB.

Where did the '3' come from? Can't see any references to 'hash' or
'bucket' in the output from db_stat...


----- s n i p -----
[papadoc.root]# db4.2_stat -h /var/lib/ldap -d /var/lib/ldap/objectClass.bdb 53162 Btree magic number.
9 Btree version number.


db_stat clearly says this is database is using a Btree. So just repeat the Btree calculation and ignore any reference to hashing.
Somewhere along the line our performance testing showed that Btrees outperformed hashing for index lookups, so the code was changed and hash tables are no longer used.


I wrote a perl script (included if it's correct and someone
needs something similar) that does the calculation for me...

Running this on my system tell me to use a cache size of
114Mb (I used 52Mb previosly which might explain things!!).

Provided this value is correct, what if I enter '150Mb' as
cache, and i get a boost in subscriptions (I'm using this
db as email/user db) and 150Mb isn't enough!? How can I
grow the cache 'live', without reloading the db? Is this
possible?


It say that I can use 'db_stat -m' to "check how well the
database cache is performing"... Sure, but what am I'm looking at!?


Sorry for all these questions about something that isn't
OpenLDAP but rather BerkeleyDB, but their documentation
(conserning DB_CONFIG and db_stat) really sucks. It's even worse than the OpenLDAP one :)

If you (someone) help me out here, I promise I'll write
down something fancy that people can actually understand :)



I'm not sure why you have such difficulty comprehending the docs, but the BDB documentation has always seemed perfectly clear to me. Maybe you're just not comfortable with technical jargon in English, and would have an easier time with docs written in your native language. I know the OpenLDAP project is always happy to accept help in translating our docs; SleepyCat would probably welcome such assistance as well.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support