Re: mdb_dbi_open and threads

Sorry for the late reply, I was on vacation.

Klaus wrote:
"I am still unsure what you are trying to achieve. If you are in a read transaction and discover that your database does not exist, what can you do anyway? You cannot create the database at this point, since it is a write operation."

When I discover a dbi does not exist in a read transaction, I can assume it to be the same as a dbi which is empty and create it when and only when there is a write-request into that dbi later.

Howard wrote:
"If you want to use dbi_open in multiple threads then put it in your own wrapper function, protected by a mutex."

This defeats the whole purpose of parallel reads.

Hallvard wrote:
"Anyway, just forget about being clever with DBIs. LMDB does not support DBI cleverness"

There is nothing clever about opening a dbi on the fly. Its a normal requirement especially when the database handles multiple clients.

Imagine a server database listening in on clients.
The server issues a write request from Client A and dbi_open ABC and is about to write data.
At the same time in another thread, it issues a read request from Client B to the same dbi ABC but is
unable to dbi_open ABC.

Isn't this scenario one of the basic requirements of a database listening in on clients.

On Mon, May 22, 2017 at 6:19 PM, Howard Chu <hyc@symas.com> wrote:

Hallvard Breien Furuseth wrote:

On 22. mai 2017 14:00, Howard Chu wrote:

Muhammed Muneer wrote:

Hallvard wrote

"With threads 1 and 2 coexisting? When thread 2 called mdb_dbi_open(),
thread 1's prospect of using mdb_dbi_open() at all was lost."

Yeah with both coexisting. Thats what I thought.

Sorry, I should have said with _transactions_ #1 and #2 coexisting,
_transaction_ #1's prospect of using mdb_dbi_open() at all was lost.
Transaction #3 in thread #1 can use it if it stared after txn#2 ended.

Anyway, just forget about being clever with DBIs. LMDB does not support
DBI cleverness, that's one of its trade-offs for speed and simplicity.

Exactly.

If you want to use dbi_open in multiple threads then put it in your own
wrapper function, protected by a mutex.

Wait, what? mdb_dbi_open() isn't coded to handle concurrent txns
calling it, regardless of any mutexes the caller has. In particular,
it does not use nor update the environment's numdbs, so the two
transactions could end up creating the same DBI for different DBs.

Yes. This is only safe if you also close the dbi in the same txn that opened it. Which you are probably doing, if you are opening and closing on the fly.

Naturally you will also have to wrap dbi_close the same way.

If we did support this, txn_commit() of the txn which used dbi_open()
would also need the mutex.

--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/