(ITS#8264) LMDB mdb_del in a cursor causes data loss in certain situations

[malyn@strangeGizmo.com]

Full_Name: Michael Alyn Miller
Version: LMDB 0.9.14, Git head
OS: Windows 8.1 x64, NixOS 14.12 x64
URL: ftp://ftp.openldap.org/incoming/michael-alyn-miller-151005.c
Submission from: (NULL) (96.251.78.237)


I have a helper library that sits on top of LMDB and testing some of the
abstractions in that library has exposed a bug in LMDB related to deleting items
while inside of a cursor.  I have reduced the repro down to a test case that
operates directly against LMDB.

The test in question adds three groups of keys to the database (each with a
different prefix) and then deletes the third group of keys.  The deletion is
done by scanning all of the keys with that prefix using a cursor. 
mdb_cursor_del works correctly and deletes only the targeted keys, whereas
mdb_del causes nine extra keys (not in that group/prefix) to be deleted.

Note that most of my test runs (with different test data) do not exhibit this
behavior: the targeted keys, and only the targeted keys, are deleted as
expected.  Something about these specific keys and values triggers the bug.  I
assume that other keys will also trigger the bug, but these are the ones that I
have converted into an LMDB-based test.

The referenced test runs twice: once with mdb_cursor_del and once with mdb_del. 
You can mdb_dump the data before and after each deletion to see the extra keys
that are being removed.  The incorrectly deleted keys are all consecutive and
begin with 02000000000000003a; the nine keys after and including 02...3aa4...
are the ones that are incorrectly deleted.  Only keys beginning with
03000000000000003a should have been deleted (so it is interesting to note that
the nine keys immediately preceding the targeted keys were the ones that were
incorrectly deleted).

Is it valid to delete keys with mdb_del while inside of a cursor?  If not, then
ignore this bug report and I will modify my helper library.  That said, I do
expose more than just delete to my callers during cursor scans (they can mdb_put
new items during a scan), which is why my library uses mdb_del and not
mdb_cursor_del in this scenario.

I have tested this code against LMDB 0.9.14 and Git head 8b46dcc.  Please let me
know if you need any additional information.  Thank you for your time!