[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: openldap.git branch mdb.master updated. 14fb1f59c7e119a100952890f947377d7e95f135
openldap-commit2devel@OpenLDAP.org wrote:
> - Log -----------------------------------------------------------------
> commit 14fb1f59c7e119a100952890f947377d7e95f135
> Author: Howard Chu <hyc@symas.com>
> Date: Sat Jul 21 06:10:02 2012 -0700
>
> Append tweaks, page_split fixes
>
> Append mode now does no key comparisons, input must be in sorted order.
> page_split was not updating cursor parents correctly.
MDB_APPEND mode is meant to be used for bulk loading of pre-sorted keys.
(e.g., if someone wanted to write an mdb_load utility and the corresponding
mdb_dump program, you would use this mode for the loader.) New key/data pairs
are simply appended to the last page of the database, with no key comparisons
at all. (That's the new behavior.) When a page is filled, instead of splitting
the page into two new pages (with half the keys in each) it just allocates a
new page and starts appending to it. (That's the same behavior as before.) If
the parent page fills, it is split (into halves) as normal. The savings from
avoiding the memcpy's associated with the split is. The savings from avoiding
the key comparisons is another 45% on top of that.
In the context of the microbenchmarks posted earlier, with this new code our
batched sequential write speed goes up from 1,157,407 entries/sec to 2,083,333
entries/sec, nearly 3 times faster than the closest competitor, LevelDB (at
~745,000 entries/sec). It also helps slapadd -q. It's around 11-1/2 times
faster than BerkeleyDB.
We should not be hearing comments about slow backup/restore procedures any more...
> -----------------------------------------------------------------------
>
> Summary of changes:
> libraries/libmdb/mdb.c | 115 +++++++++++++++++++++++++++++++++++++++---------
> libraries/libmdb/mdb.h | 24 ++++++++++-
> 2 files changed, 117 insertions(+), 22 deletions(-)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/