[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: ldap query performance issue



Hello,

because of this,  does it make sense in a directory with > 1,000,000
people to index the sex?

thanks Meike

2013/5/23 Quanah Gibson-Mount <quanah@zimbra.com>:
> --On Thursday, May 23, 2013 4:40 PM +0000 Chris Card <ctcard@hotmail.com>
> wrote:
>
>> Hi all,
>>
>> I have an openldap directory with about 7 million DNs, running openldap
>> 2.4.31 with a BDB backend (4.6.21), running on CentOS 6.3.
>>
>> The structure of the directory is like this, with suffix dc=x,dc=y
>>
>> dc=x,dc=y
>>    account=a,dc=x,dc=y
>>       mail=m,account=a,dc=x,dc=y           // Users
>>       ....
>>       licenceId=l,account=a,dc=x,dc=y      // Licences,
>> objectclass=licence       ....
>>       group=g,account=a,dc=x,dc=y          // Groups
>>       ....
>>       // etc.
>>
>>    account=b,dc=x,dc=y
>>       ....
>>
>> Most of the DNs in the directory are users or groups, and the number of
>> licences is small (<10) for each account.
>>
>> If I do a query with basedn account=a,dc=x,dc=y and filter
>> (objectclass=licence) I see wildly different performance, depending on
>> how many users are under account a. For an account with ~30000 users the
>> query takes 2 seconds at most, but for an account with ~60000 users  the
>> query takes 1 minute.
>>
>> It only appears to be when I filter on objectclass=licence that I see
>> that behaviour. If I filter on a different objectclass which matches a
>> similar number of objects to the objectclass=licence filter, the
>> performance doesn't seem to depend on the number of users.
>>
>> There is an index on objectclass (of course), but the behaviour I'm
>> seeing seems to indicate that for this query, at some point slapd stops
>> using the index and just scans all the objects under the account.
>>
>> Any ideas?
>
>
> Increase the IDL range.  This is how I do it:
>
> --- openldap-2.4.35/servers/slapd/back-bdb/idl.h.orig   2011-02-17
> 16:32:02.598593211 -0800
> +++ openldap-2.4.35/servers/slapd/back-bdb/idl.h        2011-02-17
> 16:32:08.937757993 -0800
> @@ -20,7 +20,7 @@
> /* IDL sizes - likely should be even bigger
>  *   limiting factors: sizeof(ID), thread stack size
>  */
> -#define        BDB_IDL_LOGN    16      /* DB_SIZE is 2^16, UM_SIZE is 2^17
> */
> +#define        BDB_IDL_LOGN    17      /* DB_SIZE is 2^16, UM_SIZE is 2^17
> */
> #define BDB_IDL_DB_SIZE                (1<<BDB_IDL_LOGN)
> #define BDB_IDL_UM_SIZE                (1<<(BDB_IDL_LOGN+1))
> #define BDB_IDL_UM_SIZEOF      (BDB_IDL_UM_SIZE * sizeof(ID))
>
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> --------------------
> Zimbra ::  the leader in open source messaging and collaboration
>