Re: [LMDB] Problem with extending size on Windows

[Alain <alpic80@gmail.com>]

Howard,

Here is the current diff file with all the changes discussed.

I am using version f39bceb from November 12.

Alain

On Thu, Dec 5, 2013 at 1:28 PM, Howard Chu <hyc@symas.com> wrote:

Alain wrote:

I encountered a problem trying to change the mapsize of an existing store.

It turned out to be that the newsize argument to mdb_env_map wasn't set
correctly when coming from
mdb_env_open2.

So I had to change the call to be:
    rc = mdb_env_map(env, meta.mm_address, newenv || env->me_mapsize !=
meta.mm_mapsize);

Please use diff -u when discussing changes. Or git diff. And tell what revision you're diffing against.

(adding the || != test)

Also I had some issues with large files (4GB) and I made some small changes
which better match documentation and reported issues, but not sure if the
original code was flawed or not. It is in mdb_env_map when we set the file
pointer and EOF:

     if (newsize) {
         if ((SetFilePointer(env->me_fd, sizelo, &sizehi, 0) ==
INVALID_SET_FILE_POINTER && GetLastError() != ERROR_SUCCESS)
             || !SetEndOfFile(env->me_fd)
             || (SetFilePointer(env->me_fd, 0, NULL, 0) ==
INVALID_SET_FILE_POINTER && GetLastError() != ERROR_SUCCESS))
             return ErrCode();
     }

Again, I made those changes according to what I noticed and this might not be
the perfect final solution.

Cheers,
Alain

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/

--- ../../lmdb-git/liblmdb/mdb.c	2013-11-12 11:10:33.000000000 -0500
+++ mdb.c	2013-12-05 17:55:12.287682300 -0500
@@ -326,8 +326,13 @@
 	 *	Requires double parenthesis around 2 or more args.
 	 */
 # define DPRINTF(args) ((void) ((mdb_debug) && DPRINTF0 args))
+#ifdef _MSC_VER //MS C++ doesn't support __func__
+# define DPRINTF0(fmt, ...) \
+	fprintf(stderr, "%s:%d " fmt "\n", __FUNCTION__, __LINE__, __VA_ARGS__)
+#else
 # define DPRINTF0(fmt, ...) \
 	fprintf(stderr, "%s:%d " fmt "\n", __func__, __LINE__, __VA_ARGS__)
+#endif
 #else
 # define DPRINTF(args)	((void) 0)
 #endif
@@ -2945,8 +2950,11 @@
 	mdb_cursors_close(txn, 0);
 
 	if (!txn->mt_u.dirty_list[0].mid &&
-		!(txn->mt_flags & (MDB_TXN_DIRTY|MDB_TXN_SPILLS)))
+		!(txn->mt_flags & (MDB_TXN_DIRTY|MDB_TXN_SPILLS))) {
+		DPRINTF(("nothing to commit txn %"Z"u %p on mdbenv %p, root page %"Z"u",
+		    txn->mt_txnid, (void*)txn, (void*)env, txn->mt_dbs[MAIN_DBI].md_root));
 		goto done;
+	}
 
 	DPRINTF(("committing txn %"Z"u %p on mdbenv %p, root page %"Z"u",
 	    txn->mt_txnid, (void*)txn, (void*)env, txn->mt_dbs[MAIN_DBI].md_root));
@@ -3304,9 +3312,9 @@
 	 * Just allocate the maxsize right now.
 	 */
 	if (newsize) {
-		if (SetFilePointer(env->me_fd, sizelo, &sizehi, 0) != (DWORD)sizelo
+		if ((SetFilePointer(env->me_fd, sizelo, &sizehi, 0) == INVALID_SET_FILE_POINTER && GetLastError() != ERROR_SUCCESS)
 			|| !SetEndOfFile(env->me_fd)
-			|| SetFilePointer(env->me_fd, 0, NULL, 0) != 0)
+			|| (SetFilePointer(env->me_fd, 0, NULL, 0) == INVALID_SET_FILE_POINTER && GetLastError() != ERROR_SUCCESS))
 			return ErrCode();
 	}
 	mh = CreateFileMapping(env->me_fd, NULL, flags & MDB_WRITEMAP ?
@@ -3470,7 +3478,7 @@
 			env->me_mapsize = minsize;
 	}
 
-	rc = mdb_env_map(env, meta.mm_address, newenv);
+	rc = mdb_env_map(env, meta.mm_address, newenv || env->me_mapsize != meta.mm_mapsize);
 	if (rc)
 		return rc;
 
@@ -5352,6 +5360,8 @@
 			if (rc) {
 				if (op == MDB_GET_BOTH || rc > 0)
 					return MDB_NOTFOUND;
+				data->mv_data = d2.mv_data;
+				data->mv_size = d2.mv_size;
 				rc = 0;
 			}