Re: slapadd performance degradation from 2.3.43 to 2.4.12

[Philip Guenther <guenther+ldapdev@sendmail.com>]

On Fri, 21 Nov 2008, Howard Chu wrote:
...
> Something else came to mind. The trickle_task obviously cannot increase 
> the overall volume of I/O, so there's no good reason for it to make 
> things more than twice as slow.

Really?  Why can't a DB page be modified, written out by memp_trickle(), 
modified again, and then written out by txn_checkpoint()?  Without the 
memp_trickle(), it would have only been written out once, no?  Also, a 
page dirtied in a transaction may be written out before the transaction is 
resolved.  If the transaction is aborted, then the page is undirtied and 
wouldn't even be written as part of the next checkpoint.


So I think the rule is actually the *opposite*: calling memp_trickle() 
cannot *reduce* the overall volume of I/O, as anything that's dirtied and 
committed between the last checkpoint and the next one must be written at 
least once by the time the next checkpoint occurs.



> Since you said that your BDB cache size is large enough to contain the 
> entire DB in each case, try this: preload the LDIF into the FS cache 
> before running slapadd. (dd if=<ldif> of=/dev/null bs=1024k)

If the entire thing fits in cache, the only way memp_trickle() could help 
would be by spreading out the writes required by checkpoints.  That is, it 
helps if it writes a page *and* that page is not dirtied before it is 
forcibly written out (by checkpoint) or dropped from the cache (by space 
pressure).


Philip Guenther