RE: Searches causing disk writes

[Adam Wale <adamw@staff.iinet.net.au>]

For anyone that was interested in the fix for this, moving to shared memory resolved the issue.

-----Original Message-----
From: Philip Guenther [mailto:guenther+ldaptech@sendmail.com] 
Sent: Sunday, 30 October 2011 4:06 AM
To: Christopher Wood; Adam Wale
Cc: openldap-technical@openldap.org
Subject: Re: Searches causing disk writes

On Sat, 29 Oct 2011, Christopher Wood wrote:
> On Sat, Oct 29, 2011 at 08:22:38PM +0800, Adam Wale wrote:
> > iotop shows me that the slapd process is doing the writing, I'm trying 
> > to identify what exactly it's writing, and if there's a way I can 
> > prevent it

How do memory writes to mmap'ed file show up in iotop?  The BDB library, 
by default, uses mmap'ed filed for its shared-memory regions for its mpool 
page cache, holding locks and the transaction data structures, etc.  If a 
search requires lots of pages to be pulled into BDB's page cache in those 
mmap'ed files, will those show as writes to iotop.

If so, and those really are getting written to disk, then switching from 
mmap'ed files to SysV shared memory may be the solution.  The shm_key 
option (olcDbShmKey in back-config) lets you enable that.  Be aware that 
on many OS's, the the default system limits for shared memory size were 
fairly low, so you may need to bump that up.  On Linux take a look at 
/proc/sys/kernel/shmmax and/or the kernel.shmmax sysctl in 
/etc/sysctl.conf


> In that case, strace/ltrace/dtrace/etc. on the slapd process will show 
> you what the process itself is doing.

strace may confirm the above to be the problem by the *absence* of 
syscalls writing data.  dtrace might be able to show it directly...


Philip Guenther