[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: write-scaling problems in LMDB



Luke Kenneth Casson Leighton wrote:
On Mon, Oct 20, 2014 at 3:25 PM, Howard Chu <hyc@symas.com> wrote:
The timings on the left are for the writer process; the reader process
is on the right. You were right, the number of writers is largely
irrelevant because they're all waiting most of the time. There's a
huge jump in System CPU time from 1 writer to 2 writers, because with
only 1 writer the mutex is uncontended and essentially zero-cost. As
more writer threads are added, the System CPU time will rise.

  by some 15 to 20% or so, from 2 writers going up to 32 writers.
which is not unreasonable - nothing like what i saw (CPU usage
dropping to 40%, loadavg jumping to over 8x the number of cores).

  ok... what's the number of writes being carried out in each
transaction?  the original scenario in which i saw the issue was when
there was one (!!).  the design at the time was not optimised for
batches of reads followed by batches of writes.

This is 1 write per txn.

   a single write per transaction would mean quite a lot more mutex usage.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/