Re: dynamic group perfs

[Jephte Clain <jephte.clain@univ-reunion.fr>]

2013/5/7 Quanah Gibson-Mount <quanah@zimbra.com>:
> --On Tuesday, May 07, 2013 9:36 AM +0400 Jephte Clain
> <jephte.clain@univ-reunion.fr> wrote:
>
>> It needs more than 30 seconds to build the dynamic group! and this is the
>> case every time I do the search
>>
>> For the sake of it, I made a static group with 45000 member, and it takes
>> 0.037 seconds for the search
>>
>> So my question is: is it normal for the dynamic group to be so slow to be
>> built? is there something I can do?
>
>

hello. thanks for your answer

> What about the second time you run a query on a subtree?
it's the same every time, so it's not a cache problem

> For back-hdb, the
> first time it has load a bunch of data into the idl cache IIRC.  Also, you
> do not show in your config if you have actually indexed the attributes to be
> used by the dynamic group.  This of course would be a requirement.  For
> example:
>
> <ldap:///ou=users,ou=cas,dc=univ-reunion,dc=fr??one?(uid=*)>
>
> pres index on uid
>
> <ldap:///ou=webaccounts,dc=univ-reunion,dc=fr??one?(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled)))>
>
> runUnivCategorie -> eq index
> runUnivAuthorizaion -> eq index

well, all these attributes have an eq index, and I assumed an eq index
would imply the pres index. maybe that's the problem. I will try it on
friday, it's not possible right now.

you may have noticed in the examples I gave in the first mail: the
individual searches by themselves are quite fast.

$ time ldapsearch (...) -b ou=users,ou=cas,dc=univ-reunion,dc=fr
"(uid=*)" dn >/dev/null
real    0m1.025s

$ time ldapsearch (...) -b ou=webaccounts,dc=univ-reunion,dc=fr
"(&(uid=*)(runUnivCategorie=inconnu)(!(runUnivAuthorization=webaccounts:disabled)))"
dn >/dev/null
real    0m0.043s

$ time ldapsearch (...) -b ou=groups,ou=eduspot,dc=univ-reunion,dc=fr
"(cn=users)" member >/dev/null
real    0m34.334s

what I don't understand is how 1.025 + 0.043 = 34.33

or perhaps 1.025 is very slow by openldap standards, and the reason is
that there is only an eq index on uid?

but I didn't set a pres index on uid because in the branches above,
nearly 100% of the objets beneath have the uid attribute, and the
openldap admin guide says:

----------------------------- 8< -----------------------------
21.2.3. Presence indexing

If your client application uses presence filters and if the target
attribute exists on the majority of entries in your target scope, then
all of those entries are going to be read anyway, because they are
valid members of the result set. In a subtree where 100% of the
entries are going to contain the same attributes, the presence index
does absolutely NOTHING to benefit the search, because 100% of the
entries match that presence filter.

So the resource cost of generating the index is a complete waste of
CPU time, disk, and memory. Don't do it unless you know that it will
be used, and that the attribute in question occurs very infrequently
in the target data.

Almost no applications use presence filters in their search queries.
Presence indexing is pointless when the target attribute exists on the
majority of entries in the database. In most LDAP deployments,
presence indexing should not be done, it's just wasted overhead.
----------------------------- 8< -----------------------------

Is dynamic group a special case where a presence index is required?

thanks in advance for any insights

>
> --Quanah
>
>
> --
>
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> --------------------
> Zimbra ::  the leader in open source messaging and collaboration



--
cordialement,
Jephté Clain
Direction des Systèmes d'Information
et des Usages Numériques - 2IG
Tél. 0262 93 86 31
Fax. 0262 93 81 06