Re: write-scaling problems in LMDB

[Howard Chu <hyc@symas.com>]

Luke Kenneth Casson Leighton wrote:
> On Mon, Oct 20, 2014 at 1:53 PM, Howard Chu <hyc@symas.com> wrote:
> > My experience from benchmarking OpenLDAP over the years is that mutexes
> > scale only up to a point. When you have threads grabbing the same mutex from
> > across socket boundaries, things go into the toilet. There's no fix for
> > this; that's the nature of inter-socket communication.
>
>   argh.  ok.  so... actually.... accidentally, the design where i used
> a single LMDB (one env) shared amongst (20 to 30) processes using
> db_open to create (10 or so) databases would mitigate against that...
> taking a quick look at mdb.c the mutex lock is done on the env not on
> the database...
>
>   sooo compared to the previous design there would only be a 20/30-to-1
> mutex contention whereas previously there were  *10 sets* of 20 or 30
> to 1 mutexes all competing... and if mutexes use sockets underneath
> that would explain why the inter-process communication (which also
> used sockets) was so dreadful.

Note - I was talking about physical CPU sockets, not network sockets.
>   huh, how about that.
>
> do you happen to have access to a straight 8-core SMP system,

No.

> or is it
> relatively easy to turn off the NUMA architecture?

I can probably use taskset or something similar to restrict a process to a 
particular set of cores. What exactly do you have in mind?

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/