RE: (ITS#2610)

[hyc@highlandsun.com]

Given the description of the problem, I'm not sure a software fix is
appropriate. The issue is that a particular thread has some locks held and is
committing a transaction to disk. At this point, there's enough I/O
accumulated that the fsync() that flushes the disk buffers will take a
non-trivial length of time. This thread can do nothing further until the
fsync() finishes.

In the meantime, other threads come along and attempt to acquire these locks.
After some number of spinlock attempts fail, these threads give up and call
sched_yield() to let some other thread run. The problem is there are no other
threads that can make any progress, because they're all waiting for the same
locks, and the lock in question won't be released until the fsync()
completes. So all of these threads yield to each other repeatedly, and CPU
utilization hits 100%.

It seems to me that there's no real harm being done. If any work comes along
that the system can process, then that work will soak up the CPU time while
all these threads are busily yielding. The fact that these threads are just
calling yield() ad infinitum indicates that the CPU just doesn't have
anything better to do.

Your best bet would be to add a second disk drive, as already recommended in
the BDB docs and the OpenLDAP FAQ. Keep the transaction logs on a separate
disk from the actual database files; that will avoid thrashing the disks and
will dramatically improve I/O performance, thus allowing fsync() to complete
sooner, and preventing so many delays from building up.

If you're experimenting with HEAD or RE22 you can additionally try using
shared memory for the BDB environment. Use "shm_key <arbitrary integer key>"
in the bdb config of slapd.conf to enable it. You will still need separate
disks for log vs data to get the best performance. This option is not yet
documented in slapd-bdb(5) since it hasn't yet proven to be of significant
usefulness.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

> -----Original Message-----
> From: owner-openldap-bugs@OpenLDAP.org
> [mailto:owner-openldap-bugs@OpenLDAP.org]On Behalf Of edg72@home.nl

> Jong,
>
> (sorry for this huge post)
>
> I've tried OpenLDAP HEAD, including Howard's patch, but I
> don't see any
> improvement when running test008.  The test was executed on
> an Athlon 1GHz
> with a 40GB UDMA IDE disk running under SuSE Linux 8.2, glibc
> 2.3.2. My guess
> is that the problem gets worse on slower hardware.