Re: lmdb memory usage when writing lots of data with memorymap option

[Howard Chu <hyc@symas.com>]

Luc Vlaming wrote:
> Hi,
>
> Currently I am creating support for using LMDB as a new storage backend for
> one of our products.
> At the moment I am testing import bulk data into lmdb using transactions that
> span a single record of 10MB. The total db size afterwards is 5GB. I also
> tested with records of 1MB.
>
> I noticed a very odd thing: when using the MDB_WRITEMAP option, memory usage
> grows very quickly and linear with the amount of data stored into the
> database. (memory usage ends up a bit higher than 5GB). when not using
> MDB_WRITEMAP, however, memory usage stays very low. Does anyone have a
> suggestion what might be wrong and what causes such different behaviour with
> and without using the memorymap option?

There is nothing wrong. It is simply writing to the shared memory map.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/