[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: slow adds of member attribute in large groups

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Paul B. Henson

> I'm working on implementing groups in our directory server. We have a
> large number of groups, many with a large number of members. I've
> noticed that when populating a large group it starts taking longer and
> longer to add a member. For reference, I've included my current

> Is this degradation in performance to be expected?

Yes. The way things work, every change you make causes the entry to be
completely read and rewritten. As the entry gets larger, each change gets
slower. Indexing compounds the problem because the index is regenerated for
the entire attribute, not just for the changed values. So when you add a
member to a 20,000 member group, it generates 20,001 index values, not just
the 1 that you added.

I had a patch in 2.2alpha to avoid the wasted indexing work, but it was
backed out because it could cause valid index keys to be removed
unintentionally, and I didn't have time to fix it before 2.2beta.

> Is there some
> further tuning I can do to make this more efficient? It is annoyingly
> slow and will take possibly an unacceptably long amount of time to
> initially create and maintain the population of some of the larger
> groups. it took over 21 hours just to populate this one group.

Two suggestions:
1) turn off indexing when you're going to be doing bulk updates like this

2) don't add members to groups one at a time. Add all your members in one
operation.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support