RE: slapd 100% CPU usage !

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Dan Rossi

> i've questioned the same problem and no good reason for it
> yet , i suggest its a bug , or a config error

Most likely config.
>
> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Jehan PROCACCIA

> Hello
>
> I am running openldap 2.1.9, with bdb-backend (4.1.24). After starting
> slapd, it takes nearly all the CPU usage :-( .
>
> $ top
> CPU states: 50.1% user, 49.8% system,  0.0% nice,  0.0% idle
> Mem:   255300K av,  246444K used,    8856K free,       0K
> shrd,   19044K
> buff
> Swap:  265032K av,   30992K used,  234040K free
>    124604K
> cached
>
>    PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
>   1939 ldap      25   0  4260 4260  3080 R    96.0  1.6   5:43 slapd
>
> I am wondering what slapd is doing so I run a strace:
>
> $ strace -p 1939
> sched_yield()                           = 0
> sched_yield()                           = 0
> sched_yield()                           = 0
> ....
> infinite lines like these .

This means something is unable to get a resource that it needs. In this case,
probably the BDB library can't get enough memory to finish an operation.

> I did recreate my base from a ldif file (extracted with slapcat from a
> 2.0.25 openldap ldbm with berkeley database) and inserted
> with slapadd .
>
> As it first took nearly an hour to do the slapadd, I did as
> recommended
> in the list, create a DB_CONFIG file to optimise slapadd operation:
>
> [root@corbeau /var/lib/ldap/int]
> $ cat DB_CONFIG
> #set the logfile size to 100MB.
> set_lg_max 104857600

I usually leave the logfile size at the default of 10MB. There's nothing
gained by changing this.

> #set the in-memory log buffer size
> set_lg_bsize 262144

I set this to 2MB. The larger the better, when using DB_TXN_NOSYNC. Within
limits, of course.

> #temporary while we're slapadding the database
> set_flags DB_TXN_NOSYNC
> #set the (per db?) cachesize to 0GB + X bytes, split into N
> pieces of memory
> set_cachesize 0 209715200 2

You're telling BDB to use 200MB of cache and you only have 256MB of system
memory, and 244MB is already in use somewhere else. slapd didn't get it, it's
only got 4MB (1.3% of total, according to your status output). Most likely
the BDB library is trying to malloc that 200MB of cache and failing, and
trying again. and again....
>
> Now I wonder if my problems comes from here ? I also see that
> we can put
> these BDB options in slapd.conf file (cachesize <integer>, checkpoint
> <kbyte> <min> , dbnosync ..) however they doesn't seem to be
> taken into
> account while slapadding !? I really needed to create the
> DB_CONFIG file
> to reduce time of insertion from 50 minutes to 45 seconds !.
>
> What causes slapd to use 100% of CPU?
> How should I optimize BDB performance (from DB_CONFIG file or
> slapd.conf) ?

For the most part the slapd.conf options don't overlap the DB_CONFIG options,
that's a wasteful duplication of effort. There are exceptions, like dbnosync
which just uses DB_TXN_NOSYNC. The cachesize option sets the size of
back-bdb's in-memory entry cache. This is different from the BDB library's
cache. The checkpoint option exists in slapd.conf because there is no
automatic checkpoint keyword for DB_CONFIG. In general you need to configure
both files to get optimum performance.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support