Re: LMDB vs FastDB

[Howard Chu <hyc@symas.com>]

Tobias Oberstein wrote:
> Hello,
>
> I have read the - very interesting - performance comparison
> http://symas.com/mdb/microbench/
>
> I'd like to ask if someone did benchmark LMDB (and/or the others)
> against http://www.garret.ru/fastdb.html
>
> FastDB is an in-memory ACID database that works via shadow paging, and
> without a transaction log.

OK, like LMDB it uses shadow root pages. I think the similarity ends there.
It is a relational database with an ASCII query language, while LMDB is 
strictly a key/value store. That automatically means for simple get/put 
operations LMDB will be orders of magnitude faster (just as it is so much 
faster than SQLite3 and SQLite4).

It makes the flawed assumption that many main-memory databases do, that all 
RAM is the same speed. It uses T-trees as its underlying data structure. LMDB 
uses B+trees because time and again research shows that B+trees are more 
efficient, even when all of the data fits into main memory. This is the same 
mistake the MemSQL guys make, as well as MySQL NDBCluster.

See these papers, for example:
http://resnet.uoregon.edu/~gurney_j/jmpc/skiplist.html
http://www.vldb.org/dblp/db/conf/vldb/RaoR99.html

The reality is that computer architectures are hierarchical in nature. A CPU 
has L1, L2, and possibly L3 cache, each of which is progressively slower than 
the previous. Then it gets to local RAM, which is slower still. Then in a 
multiprocessor machine, there is NUMA, memory residing on remote nodes, which 
is again slower still. B+trees are inherently hierarchical and naturally 
suited to the reality of system design. All other approaches, including the 
recently popular LSM trees (as used in e.g. Google LevelDB, SQLite4, and 
various other recent databases) are all inherently inferior because of this 
fundamental fact of computer architecture.

FastDB also appears to use locking, while LMDB is MVCC and readers require no 
locks, so even with all of the other disadvantages out of the way, LMDB will 
scale better across multiple CPUs.

FastDB is also written in C++ which is another inherent disadvantage compared 
to LMDB which is pure C.

You could adapt the LevelDB microbenchmarks to test it but ultimately I 
believe it would be a waste of time.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/