[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: BerkeleyDB performance on Linux
Howard Chu wrote:
The caching problem is particularly aggravated on Linux because the
memory manager doesn't give program pages higher priority than cache
pages. So when your system is tight on memory, the kernel will start
swapping program data pages before it starts reclaiming buffer cache
pages, and application performance plummets. (Possibly that indicates a
kernel bug, or at least a misfeature.)
Thanks to Rik van Riel for enlightening me here about
/proc/sys/vm/swappiness. The default setting on the system was 60 (range
0-100) but setting it down to 10 reduced the problem considerably. With a
setting of 10 only 300MB of the slapd process got swapped out, and for the
most part the swapd's were idle after that. Total throughput is around 11,200
authentications per second. Not quite as fast as the Direct I/O case, but
much much better than before. Some time apparently is still lost due to
swapping - the swap in use decreases slowly, indicating that the swapped out
data pages are still needed. I suppose running with swappiness=0 would
eliminate that, will try that after the current swappiness=10 test completes.
(There's a downside to the O_DIRECT patch - it requires every buffer
allocation that BDB makes to be overallocated by 512 or 4096 bytes, so that
the buffer can be properly aligned. But it certainly yields the best
performance overall.)
Note that you must configure BerkeleyDB with --enable-o_direct to enable
the support, and you must add "set_flags DB_DIRECT_DB" to your DB_CONFIG
to enable it in a particular environment.
With this patch, a slapd that occupies 6.8GB on a system with 8GB of RAM
can run continuously without swapping, delivering a sustained 11,500
authentications per second. Without the patch, swapping starts when the
process hits the 4.5GB mark (because over 3GB of buffer cache is in
use), and performance drops to only *hundreds* of authentications per
second.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/