Re: (ITS#7789) Unreliable mdb_env_set_mapsize()

[h.b.furuseth@usit.uio.no]

On 25/07/14 12:54, Howard Chu wrote:
> h.b.furuseth@usit.uio.no wrote:
>> Another problem:
>>
>> With an MDB_RDONLY environment on Windows, env->me_mapsize
>> does not seem to affect the actual map size or vice versa.
>> mdb_env_map() does
>>         /* Don't set explicit map size, use whatever exists */
>>
>> If the user sets a mapsize bigger than the current file,
>
> That is impossible on Windows. The OS will not allow a mapsize bigger
> than the filesize.

Clarification: If the Windows user sets MDB_env.me_mapsize
bigger than the file size, then as far as I can tell the read-
only MDB_env will think the map is bigger than it is.  Then:

>> and another process resizes the map and grows the DB into
>> the new size, then the RDONLY process will not discover
>> in mdb_txn_renew0() that its map is too small. (...)
>
> That change was made to accomodate opening of an mdb_copy'd DB file in
> read-only mode. Since mdb_copy only copies the used pages, the resulting
> file will be smaller than the configured mapsize. If you attempt to map
> this file as-is with the configured mapsize, it will fail. You would
> have to resize the file, but since the env is opened read-only, you
> should not make any such change to the file.
>
> We could alternatively change mdb_copy to explicitly set the filesize to
> match the mapsize at the end of the copy, and then this particular hack
> wouldn't be needed.

With the several ways to play games with mapsize, the hack will
be needed anyway.  So I expect lmdb should implement and document:

   On Windows, read-only environments ignore the configured mapsize
   and instead use the file size as of the last mdb_env_set_mapsize()
   or mdb_env_open().  MDB_envinfo.me_mapsize will show that size.
   [So does MDB_env.me_mapsize.]
   For a bigger map, grow the file first - e.g. by opening the
   environment with write access and a bigger mapsize.

Setting filesize=mapsize: MDB_env.me_mapsize or MDB_meta.mm_mapsize?
(I don't really care though, since the "set mapsize=filesize"
suggestion addresses my original concern.)

OTOH mdb_copy seems the natural place for a "shrink mm_mapsize"
option, since it can only grow in an existing file.  E.g. set it
to the minimum possible or to MDB_env.me_mapsize and let the user
grow it to a bigger size later if needed.