Re: LMDB data corruption/alignment issues on ARM

[Martin Lucina <martin@lucina.net>]

martin@lucina.net said:
> hyc@symas.com said:
> > >My question is: Given that I'm storing C structs directly in LMDB, I need
> > >to get at least word-aligned pointers back from LMDB in order to be able to
> > >access the data safely.
> > >
> > >I've not found anything in the documentation or the source about being able
> > >to tweak alignment of key and data values; I *think* that all I need is an
> > >option where LMDB would guarantee a minimum word (4 byte in this case)
> > >alignment of data.
> > >
> > >How hard would this be to implement? Would you consider such a feature?
> > 
> > LMDB guarantees 2-byte alignment of keys, and data is generally
> > stored contiguously with keys. If you want 4-byte alignment, it's
> > your responsibility to pad your keys to 4-byte boundaries. If you
> > pad appropriately, your alignment will be preserved.
> 
> In this case the keys are size_t (4 bytes on the target) and I'm using
> MDB_INTEGERKEY. I'm not sure what you mean by padding the keys to 4-byte
> boundaries given that LMDB copies my data into its memory map and I have no
> control over where it gets placed.

Ah. What you forgot to mention (and I tracked down in a Reddit comment
somewhere) is that the (key + data) size (and not just the key size) should
be a multiple of the required alignment.  The problematic records in
question are using VLAs to store variable length strings; if I pad the
allocated space to the nearest word boundary that seems to fix the issue.

Martin