Re: question about use of buffers in lmdb

[Howard Chu <hyc@symas.com>]

Luke Kenneth Casson Leighton wrote:
> howard, et al, hi,
>
> i notice that LMDB stores data in an "append only" manner (from the
> Q&A).  i was thinking of recommending that in python the buffers be
> handed around with *direct* and persistent pointers to the data within
> the LMDB shm/memory-map.
>
> the reason that i believe this will be successful is because the
> circumstances are rather unusual: once data is entered (added) it is
> *never* overwritten, updated, changed, or modified.  there simply is
> no need [in this application].  so there will only ever be insertion
> (once and only once) or controlled and monitored deletion (by age of
> record).
>
> under these circumstances is it perfectly reasonable to pass around
> the pointers to the buffers returned from value look-ups
> *indefinitely*?  if so that would be awesome.

Any time a new value is inserted into an existing page, that page's address 
will change due to COW. Any time an old page gets sufficiently old that no 
other txn points to it, it becomes eligible for reuse - thus completely 
unrelated data could get onto it. So unless you're keeping very close tabs on 
your application's key+data sizes and know how full a page got, you can't rely 
on particular addresses staying valid.
> l.


--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/