Re: Issues arising from creating powerdns backend based on LMDB

[Mark Zealey <spam@markandruth.co.uk>]

On 23/08/13 14:02, Howard Chu wrote:
> Mark Zealey wrote:
> > > 4) Creating database with non-sequential keys is very bad (on 4gb
> > > databases, 2* slower than kyoto - about 1h30 and uses more memory).
> >
> > This was actually a typo - kyoto only takes about 20 minutes to generate
> > it so 4* slower. However, using a commit every 1m inserts (and because
>
> Try again with commit every 100K inserts.

Thanks, that's much quicker again - for commit every 30k or 100k inserts 
time is about 12m30s. This is probably more limited by out pulling the 
data now. Of course this is still using memdisk

I've found another weird - I have now converted the database to use 
duplicates. Typically when I do mdb_cursor_get(... MDB_NEXT ) it will 
set the key and value but I've found 1 place so far where I do it and on 
the duplicate's second entry the value is set but the key is empty. 
However in other cases with duplicates both key and value are returned. 
Also, using MDB_NEXT_DUP seems to return key/value in all cases. Could 
this be another feature caused by a page boundary ? (I'm assuming here 
that MDB_NEXT is meant to go through dups as well although the docs 
don't specifically say that)

Mark

Mark