LMDB fragmentation problem (MDB_MAP_FULL)

[Ross MacGregor <rmacgregor@wurldtech.com>]

I am running tests of LMDB and getting the MDB_MAP_FULL error adding 
entries to the database.

I am filling a database to 80% capacity and deleting and adding entries 
so that the database fluctuates between 60% and 80% capacity.

If I add entries that are greater than a single page size (4096) then 
the database free list very quickly becomes fragmented and it dies with 
a MDB_MAP_FULL error even though the database is less than 80% full. By 
very quickly I mean running my test application for a few seconds with a 
small database of 8 Mb. If I increase the database size to 1 Gb I get 
the same results within a minute or two.

I've even encountered this with a 1G database and entries that fit into 
a single page, but it takes significantly longer.

I am considering performing a defrag of the database if I get the error 
by using the mdb_env_copy2 function. This can take on the order of 1 
second to complete with the 1G database. This seems like a horrible 
workaround though and I may need to replace LMDB with something less 
fast but more robust.

Is there any other way to perform a defragmentation of the database?