[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: LMDB crash consistency, again

Hallvard Breien Furuseth wrote:

Catching up with old mail...

On 20/10/14 12:44, Howard Chu wrote:

This paper
https://www.usenix.org/conference/osdi14/technical-sessions/presentation/zheng_mai

describes a potential crash vulnerability in LMDB due to its use of
fdatasync instead of fsync when syncing writes to the data file. The
vulnerability exists because fdatasync omits syncs of the file metadata;
if the data file needed to grow as a result of any writes then this
requires a metadata update.


Looks like an OS bug.  fdatasync() should not break data integrity, it
may only skip metadata which are unneeded for retrieving the data.  So
size changes are synced.  So say the Posix spec and the Linux manpage.
Ah good point. If you check out their slides, #103 of 106 asks the question; the only failure they found in LMDB occurred on ext3 (and not on XFS) so we may just chalk this up to a flaw in ext3 instead.
Given that ext3 has already been superseded by ext4, this result of theirs may not be all that useful in the real world. We already have disrecommended ext3 for performance reasons, perhaps we should just note this and move on. 

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/