Re: Large "dynamic" groups and performance

[Al <afrunning@gmail.com>]

>> cachesize 500000
>
>> ###### DB_CONFIG
>
>> set_cachesize 2 0 1
>> set_flags       DB_TXN_NOSYNC
>> set_lg_bsize 5097152
>> set_lg_max 50485760
>
>
> You never state the size of your database (how many dns), or the disk size of your database (du -c -h *.bdb), so there is no way to tell if these settings are in any way valid.

Thanks for getting back to me, I'm new here :) - here is some more data:

 Approximately 450,000 dns - about 200+/- are groups, the rest are users.
du output:

163M	cn.bdb
92M	dn2id.bdb
415M	id2entry.bdb
3.7M	objectClass.bdb
16M	uid.bdb
16M	uniqueMember.bdb
703M	total

> I don't see a checkpoint setting for slapd.conf/cn=config, and I don't see an idlcachesize setting.

I added those as well, no noticeable change in performance:

cachesize 500000
idlcachesize 500000
checkpoint 10000 15

> Also, you are definitely not using "dynamic" groups in the OpenLDAP sense of the word, although they would probably perform significantly better for you.

Yes, I understand that - thats why I put it in quotes.  I looked into
using OpenLDAP dynamic lists, but I think I'm limited by the fact that
some of our systems requiring these groups need to do searchs off of
it based on the dynamic membership (and from what I can tell, its not
possible to use it that way), ie they need to search for
(uniquemember=cn=xxxx,cn=users,...) on my group section of the tree.

I'll openly admit some of the values I have been picking for caching
and checkpoint are somewhat arbitrary.  I've been trying many
different values and have yet to settle on any that work well.  I'll
gladly try any recommendations.

Thanks again, I appreciate your response.

Al