Re: LMDB write performance on Linux

[Quanah Gibson-Mount <quanah@zimbra.com>]

--On Tuesday, September 10, 2013 4:03 PM -0700 Howard Chu <hyc@symas.com> 
wrote:

> Quanah noted a few times in IRC that LMDB write performance was a lot
> slower after upgrading from Linux 2.6.x to 3.x. I've also observed some
> major slowdowns on an Ubuntu 3.5.0-23 kernel whenever the dirty page
> flusher is active. Apparently this is a known issue introduced in the 3.0
> kernel series
>
> http://lwn.net/Articles/486311/
>
> I'm told that this has been fixed as of Linux 3.10. Unfortunately our
> most recent round of benchmarks has all been run on our 3.5.0-23 system.
> The SLAMD results we obtained for LMDB Modify throughput (tested back in
> February this year) are only about 60% of the speed we expected.
>
> http://lkml.indiana.edu/hypermail/linux/kernel/1309.1/01585.html

I can confirm that switching to the 3.10 kernel on my ubuntu server 
resolved my perf issues with slapadd and mdb on ext4.  Instead of 
approximately 28 hours to load my large LDIF, it now takes 66 minutes. 
This brings it much more in line with ext2, which takes 59 minutes.

--Quanah

--

Quanah Gibson-Mount
Lead Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration