On-demand page mapping in LMDB

[Howard Chu <hyc@symas.com>]

The vl32 branch on gitorious has some tweaks for LMDB to mmap on individual 
page accesses, instead of mmap'ing the entire DB file all at once. This is 
primarily to support usage on 32 bit machines, where the address space is too 
small to accomodate multiple DBs mapped in competely, all at once.

https://gitorious.org/mdb/mdb/source/69d7cb8d44e04f02d8d0c923ae71fbaaa9f42f3a:

There's a significant performance hit since almost every page reference 
results in an mmap system call. But it also has a much smaller memory footprint.

While the code passes simple tests, it is also broken in one significant 
aspect: Currently, advancing a cursor to a new page causes the previous page 
to be unmapped. If there are two cursors open on the same DBI in the same 
transaction, it's possible that the other cursor will still be pointed to the 
old page, and the unmap will break it. There's currently no check to prevent 
this, and since read txns don't track all of their open cursors, there is no 
way to check (unless we add refcounts to this array, which would be clumsy).

The solution I'm considering is to get rid of the per-txn array of mapped 
pages that I added here, and just operate on the cursor stacks directly. In 
that case, every cursor will have its own set of mapped read-only pages, and 
nothing that happens on one cursor will have any effect on any other cursor. 
(For read-only pages. For dirty pages, the usual cursor/page fixups apply as 
always.)

--
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/