Re: Re: LMDB usage on windows - to much memory needed

[Christian Kratzer <ck-lists@cksoft.de>]

Hi,

On Thu, 29 Jan 2015, Frank Offermanns wrote:

> Hi,
>
> thanks for the information, but unfortunately this information is not
> helpfull for me.
>
> Is there a way to configure LMDB on Windows 32 bit, so that a database can
> be larger?
> Or is it mandantory to use 64 bit with lmdb (at least on windows file
> systems) when handling larger data?

You need 64 bits on any kind of OS to support multi gigabyte databases.

No 32 bit OS will allow a single process to memory map much more than 1 or perhaps 2 GB of address space.

Greetings
Christian



> Regards,
> Frank
>
>
>
>
> Von:    Kristoffer Sjögren <stoffe@gmail.com>
> An:     Frank Offermanns <Frank.Offermanns@caseris.de>,
> Kopie:  openldap-technical@openldap.org
> Datum:  28.01.2015 16:32
> Betreff:        Re: LMDB usage on windows - to much memory needed
>
>
>
> Regarding size.
>
> "If an application specifies a size for the file mapping object that is
> larger than the size of the actual named file on disk and if the page
> protection allows write access (that is, the flProtect parameter specifies
> PAGE_READWRITE or PAGE_EXECUTE_READWRITE), then the file on disk is
> increased to match the specified size of the file mapping object. If the
> file is extended, the contents of the file between the old end of the file
> and the new end of the file are not guaranteed to be zero; the behavior is
> defined by the file system. If the file on disk cannot be increased,
> CreateFileMapping fails and GetLastError returns ERROR_DISK_FULL."
>
> http://msdn.microsoft.com/en-us/library/windows/desktop/aa366537%28v=vs.85%29.aspx
>
> On Wed, Jan 28, 2015 at 3:27 PM, Frank Offermanns <
> Frank.Offermanns@caseris.de> wrote:
> Hello,
>
> we would like to change our backend from hdb to mdb.
>
> I did some initial tests and found out the following:
> When setting maxsize to 1.4 GB, the size of the database on the file
> system is about 1.4 GB. I thought I read that the maxsize has nothing to
> do with the actual physical size needed. (If I have only 10 MB of data my
> database should be about 10 MB).
> Even worse is the fact, that slapd.exe then needs about 1.5 GB virtual
> size memory.
> And if I use 32 Bit slapd.exe the process will crash when reaching 2 GB
> virtual size. (as every 32 bit process will do)
>
> Now my conclusion is, that the things described in the mdb paper are only
> valid for unix/linux, because windows uses another memory system. Is this
> correct? Or is there a way I haven't found to configure OpenLDAP on
> windows so that it does not need so much RAM.
> The size on the disk does not matter for us, but since we use only 32 bit
> slapd.exe the RAM does matter.
>
> Best regards,
> Frank

--
Christian Kratzer                   CK Software GmbH
Email:   ck@cksoft.de               Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0       D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9       HRB 245288, Amtsgericht Stuttgart
Mobile:  +49 171 1947 843           Geschaeftsfuehrer: Christian Kratzer
Web:     http://www.cksoft.de/