[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: OpenLDAP high CPU usage when performing mass changes

Jeffrey Crawford wrote:

I'm sure this would have been immensely helpful If I had mentioned that this
is running on FreeBSD, just in case that changes answers
Indeed. I don't recall, does FreeBSD use glibc as their system C library? If not, then tcmalloc may not be relevant. 


On Fri, Mar 16, 2012 at 12:32 AM, Howard Chu <hyc@symas.com
<mailto:hyc@symas.com>> wrote:

    Jeffrey Crawford wrote:

        We are using openldap 2.4.26 with BDB 4.8 and have replication set up in
        mirror mode for our main ldap database. There are a couple of other
        replicas
        that have a subset of the data that the main cluster has but we are
        seeing the
        following behavior on all of them.

        When performing mass updates via LDAP, lets say on the order of 30,000
        entries
        being added to existing entries. We've noticed that the CPU use of the
        slapd
        instances goes through the roof (between 65% and 95% continuously),
        and seems
        to stay there until it is restarted.


    When the CPU usage goes high like that it should be pretty easy to see
    where it's going, by getting a gdb stack trace of the running process.

    At a guess, based on the minimal amount of information here, you've run
    into the glibc malloc fragmentation issue, and switching to tcmalloc might
    avoid the problem.


Fair enough doesn't look like gperftools are a package in FreeBSD so we'll
have to do a manual install but it looks like this would be the easiest and
logical first step to just "see if this fixes things"


The *first step* is to get the gdb backtrace and see WTH is going on.


Just for reference we would want to set
LD_PRELOAD=/path/to/libtcmalloc_minimal.so and not try to compile OpenLDAP
against it right?


If it appears that tcmalloc is called for, then yes, this is the way to go.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/