Re: LMDB file size, again

[Christian Sell <christian@gsvitec.com>]

well, beyond the requirement about copyability of the file, there is also the
requirement that the app coexist well with other processes that are using the
disk and not grab all disk space while running. And then, there is the
requirement that the database should still be able to grow up to the limit of
the available resources if need be.

This is a desktop app, and the database can be looked upon as a "project file".
If user copies it in a inconsistent state, it's their problem.

I have decided to implement a database growth algorithm. There is an initial map
size and a threshhold. Whenever a write transaction is started, I check that
enough space is available and enlarge the map if needed. The app may also also
request more space for a transaction if known beforehand. If the transaction
hits the MAP_FULL anyway, bad luck.

regards + thanks,
Chris

> Klaus Malorny <Klaus.Malorny@knipp.de> hat am 16. November 2015 um 11:04
> geschrieben:
>
>
> On 11.11.2015 23:51, Christian Sell wrote:
> >> You're being stupid.
> >
> > let's say I am stubborn. As I mentioned, I have the requirement that the
> > user be
> > able to copy the database file away on their USB stick (or send it via
> > email,
> > for that matter). With SQLite, that is no problem. But with LMDB under
> > Windows
> > the user is looking at a huge file, which will be unacceptable in all
> > likelihood. I also must allow the user to copy using file system tools, and
> > not
> > some custom utility.
> >
> > So I take it that LMDB is simply not designed or suitable for such a
> > scenario.
> >
>
> Maybe you should consider to write your own exporter/importer. It shouldn't be
> too hard to walk through all key/value pairs and write out the data (prepended
> by their respective lengths), plus some metadata at the beginning to recognize
> the format on import and to know the number of records (if there is no special
> end-of-file marker).
>
> Just my two cents.
>
> Klaus
>