[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssometimes
- To: openldap-its@OpenLDAP.org
- Subject: RE: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssometimes
- From: kriszyp@gmail.com
- Date: Thu, 21 Feb 2019 17:15:08 +0000
- Auto-submitted: auto-generated (OpenLDAP-ITS)
--_36998F79-0E7E-483F-8FFD-6401D532832A_
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
I see it now! I really did check even earlier this morning and wasn=E2=80=
=99t there until recently! Anyway, thank you, sorry for the noise.
Thanks,
Kris
From: Quanah Gibson-Mount
Sent: February 21, 2019 10:12 AM
To: kriszyp@gmail.com; openldap-its@OpenLDAP.org
Subject: Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssom=
etimes
Hi,
It is literally the first set of commits listed at that URL, so it's=20
clearly already there.
Hope that helps!
--Quanah
--On Thursday, February 21, 2019 4:45 PM +0000 kriszyp@gmail.com wrote:
> --0000000000000dacee05826a34a9
> Content-Type: text/plain; charset=3D"UTF-8"
>
> Thank you, I appreciate it! Do you know how often or when this will get
> into the github mirror https://github.com/LMDB/lmdb/commits/mdb.RE/0.9 (o=
r
> if I am misunderstanding how the branches are synced)?
>
> On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <hyc@symas.com> wrote:
>
>> kriszyp@gmail.com wrote:
>> > Full_Name: Kristopher William Zyp
>> > Version: LMDB 0.9.23
>> > OS: Windows Server 2012 R2, Windows 10
>> > URL:
>> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45b
>> d32ab4e8e1f2.patch
>> > Submission from: (NULL) (71.199.6.148)
>> >
>>
>> Thanks for the report and patch, added to mdb.RE/0.9
>>
>> > Calling mdb_env_set_mapsize to increase the map size, when a DB is als=
o
>> in use
>> > by other processes, when MDB_WRITEMAP is enabled (and the db file has
>> been
>> > opened with PAGE_READWRITE access), on Windows, will occasionally
>> > (seems
>> like
>> > about 1/100 attempts fails) produce an error "The requested operation
>> cannot be
>> > performed on a file with a user-mapped section open", or segfaults. Th=
e
>> error
>> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map tha=
t
>> is
>> > performed to increase the allocated file size to the map size, prior t=
o
>> > CreateFileMapping.
>> >
>> > As it turns out this is pretty easy to solve, because manually
>> > expanding
>> the
>> > file size is not necessary when calling CreateFileMapping with
>> PAGE_READWRITE
>> > access, as Windows will automatically expand the file size for us, whe=
n
>> opened
>> > with the page write access enabled. Of course, this means all processe=
s
>> must be
>> > consistent in use of MDB_WRITEMAP, but the documentation already makes
>> this
>> > explicit and clear.
>> >
>> > I believe this can be fixed by simply adding a check for MDB_WRITEMAP
>> > in
>> the if
>> > statement that calls SetFilePointer:
>> >
>> > if (!(flags & MDB_WRITEMAP) &&
>> > (SetFilePointer(env->me_fd,
>> sizelo, &sizehi, 0)
>> > !=3D (DWORD)sizelo
>> > || !SetEndOfFile(env->me_fd)
>> > || SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0=
))
>> > return ErrCode();
>> >
>> > The attached URL has the change as a patch/diff as applied to our node
>> package.
>> >
>> > I am certainly happy to just keep this change on our own branches.
>> > There
>> may be
>> > nuances of this that I might not be aware of, but it seems to be
>> > working
>> great
>> > for us and I have tested this with MDB_WRITEMAP enabled and disabled.
>> > So
>> I
>> > thought I would offer/suggest this change, as it seems like it is
>> > straightforward change to improve stability. Thank you!
>> >
>> >
>>
>>
>> --
>> -- Howard Chu
>> CTO, Symas Corp. http://www.symas.com
>> Director, Highland Sun http://highlandsun.com/hyc/
>> Chief Architect, OpenLDAP http://www.openldap.org/project/
>>
>
> --0000000000000dacee05826a34a9
> Content-Type: text/html; charset=3D"UTF-8"
> Content-Transfer-Encoding: quoted-printable
>
> <div dir=3D3D"ltr"><div dir=3D3D"ltr">Thank you, I appreciate it! Do you =
know
> h=3D ow often or when this will get into the github mirror <a
> href=3D3D"https://gi=3D
> thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB/lmdb/commi
> ts=3D /mdb.RE/0.9</a> (or if I am misunderstanding how the branches are
> synced)?<=3D /div></div><br><div class=3D3D"gmail_quote"><div dir=3D3D"lt=
r"
> class=3D3D"gmail_a=3D ttr">On Sun, Feb 10, 2019 at 3:14 AM Howard Chu <=
;<a
> href=3D3D"mailto:hyc@s=3D ymas.com">hyc@symas.com</a>>
> wrote:<br></div><blockquote class=3D3D"gmail_=3D quote" style=3D3D"margin=
:0px
> 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=3D
> 204);padding-left:1ex"><a href=3D3D"mailto:kriszyp@gmail.com";
> target=3D3D"_blan=3D k">kriszyp@gmail.com</a> wrote:<br>
> > Full_Name: Kristopher William Zyp<br>
> > Version: LMDB 0.9.23<br>
> > OS: Windows Server 2012 R2, Windows 10<br>
> > URL: <a
> href=3D3D"https://github.com/kriszyp/node-lmdb/commit/6df903907f=3D
> 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D3D"noreferrer"
> target=3D3D"_blank">=3D
> https://github.com/kriszyp/node-lmdb/commit/6df903907f5516320e9a8afce45bd
> 32=3D ab4e8e1f2.patch</a><br>
> > Submission from: (NULL) (71.199.6.148)<br>
> > <br>
> <br>
> Thanks for the report and patch, added to mdb.RE/0.9<br>
> <br>
> > Calling mdb_env_set_mapsize to increase the map size, when a DB is
> als=3D o in use<br>
> > by other processes, when MDB_WRITEMAP is enabled (and the db file
> has =3D been<br>
> > opened with PAGE_READWRITE access), on Windows, will occasionally
> (see=3D ms like<br>
> > about 1/100 attempts fails) produce an error "The requested
> opera=3D tion cannot be<br>
> > performed on a file with a user-mapped section open", or
> segfault=3D s. The error<br>
> > occurs in the SetFilePointer (or SetEndOfFile) call in mdb_env_map
> tha=3D t is<br>
> > performed to increase the allocated file size to the map size, prior
> t=3D o<br>
> > CreateFileMapping.<br>
> > <br>
> > As it turns out this is pretty easy to solve, because manually
> expandi=3D ng the<br>
> > file size is not necessary when calling CreateFileMapping with
> PAGE_RE=3D ADWRITE<br>
> > access, as Windows will automatically expand the file size for us,
> whe=3D n opened<br>
> > with the page write access enabled. Of course, this means all
> processe=3D s must be<br>
> > consistent in use of MDB_WRITEMAP, but the documentation already
> makes=3D this<br>
> > explicit and clear.<br>
> > <br>
> > I believe this can be fixed by simply adding a check for
> MDB_WRITEMAP =3D in the if<br>
> > statement that calls SetFilePointer:<br>
> > <br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0if (!(flags
> &=3D ; MDB_WRITEMAP) && (SetFilePointer(env->me_fd, sizelo,
> &size=3D hi, 0)<br>
> > !=3D3D (DWORD)sizelo<br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0
> =3D =3DC2=3DA0 =3DC2=3DA0|| !SetEndOfFile(env->me_fd)<br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0
> =3D =3DC2=3DA0 =3DC2=3DA0|| SetFilePointer(env->me_fd, 0, NULL, 0) !=
=3D3D 0))<br>
> >=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0
> =3D =3DC2=3DA0 =3DC2=3DA0return ErrCode();<br>
> > <br>
> > The attached URL has the change as a patch/diff as applied to our
> node=3D package.<br>
> > <br>
> > I am certainly happy to just keep this change on our own branches.
> The=3D re may be<br>
> > nuances of this that I might not be aware of, but it seems to be
> worki=3D ng great<br>
> > for us and I have tested this with MDB_WRITEMAP enabled and
> disabled. =3D So I<br>
> > thought I would offer/suggest this change, as it seems like it is<br=
>
> > straightforward change to improve stability. Thank you!<br>
> > <br>
> > <br>
> <br>
> <br>
> -- <br>
> =3DC2=3DA0 -- Howard Chu<br>
> =3DC2=3DA0 CTO, Symas Corp.=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
=3DC2=3DA0 =3DC2=3DA0<a
> href=3D3D"=3D http://www.symas.com"; rel=3D3D"noreferrer"
> target=3D3D"_blank">http://www.symas=3D .com</a><br>
> =3DC2=3DA0 Director, Highland Sun=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a
> href=3D3D"http://highland=3D sun.com/hyc/" rel=3D3D"noreferrer"
> target=3D3D"_blank">http://highlandsun.com/h=3D yc/</a><br>
> =3DC2=3DA0 Chief Architect, OpenLDAP=3DC2=3DA0 <a
> href=3D3D"http://www.openldap.org/p=3D roject/" rel=3D3D"noreferrer"
> target=3D3D"_blank">http://www.openldap.org/proje=3D ct/</a><br>
> </blockquote></div>
>
> --0000000000000dacee05826a34a9--
>
>
>
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>
--_36998F79-0E7E-483F-8FFD-6401D532832A_
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="utf-8"
<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc=
hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of=
fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40";><head><meta ht=
tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name=
=3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=3DEN-CA link=3Dblue vlink=3D"#954F72"><div cla=
ss=3DWordSection1><p class=3DMsoNormal>I see it now! I really did check eve=
n earlier this morning and wasn=E2=80=99t there until recently! Anyway, tha=
nk you, sorry for the noise.</p><p class=3DMsoNormal><o:p> </o:p></p><=
p class=3DMsoNormal>Thanks,<br>Kris</p><p class=3DMsoNormal><o:p> </o:=
p></p><div style=3D'mso-element:para-border-div;border:none;border-top:soli=
d #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=3DMsoNormal style=3D'bo=
rder:none;padding:0cm'><b>From: </b><a href=3D"mailto:quanah@symas.com";>Qua=
nah Gibson-Mount</a><br><b>Sent: </b>February 21, 2019 10:12 AM<br><b>To: <=
/b><a href=3D"mailto:kriszyp@gmail.com";>kriszyp@gmail.com</a>; <a href=3D"m=
ailto:openldap-its@OpenLDAP.org">openldap-its@OpenLDAP.org</a><br><b>Subjec=
t: </b>Re: (ITS#8975) Calling mdb_env_set_mapsize on a shared DB failssomet=
imes</p></div><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNorma=
l>Hi,</p><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>It =
is literally the first set of commits listed at that URL, so it's </p><p cl=
ass=3DMsoNormal>clearly already there.</p><p class=3DMsoNormal><o:p> <=
/o:p></p><p class=3DMsoNormal>Hope that helps!</p><p class=3DMsoNormal><o:p=
> </o:p></p><p class=3DMsoNormal>--Quanah</p><p class=3DMsoNormal><o:p=
> </o:p></p><p class=3DMsoNormal>--On Thursday, February 21, 2019 4:45=
PM +0000 kriszyp@gmail.com wrote:</p><p class=3DMsoNormal><o:p> </o:p=
></p><p class=3DMsoNormal>> --0000000000000dacee05826a34a9</p><p class=
=3DMsoNormal>> Content-Type: text/plain; charset=3D"UTF-8"</p>=
<p class=3DMsoNormal>><o:p> </o:p></p><p class=3DMsoNormal>> Tha=
nk you, I appreciate it! Do you know how often or when this will get</p><p =
class=3DMsoNormal>> into the github mirror https://github.com/LMDB/lmdb/=
commits/mdb.RE/0.9 (or</p><p class=3DMsoNormal>> if I am misunderstandin=
g how the branches are synced)?</p><p class=3DMsoNormal>><o:p> </o:=
p></p><p class=3DMsoNormal>> On Sun, Feb 10, 2019 at 3:14 AM Howard Chu =
<hyc@symas.com> wrote:</p><p class=3DMsoNormal>><o:p> </o:p><=
/p><p class=3DMsoNormal>>> kriszyp@gmail.com wrote:</p><p class=3DMso=
Normal>>> > Full_Name: Kristopher William Zyp</p><p class=3DMsoNor=
mal>>> > Version: LMDB 0.9.23</p><p class=3DMsoNormal>>> >=
; OS: Windows Server 2012 R2, Windows 10</p><p class=3DMsoNormal>>> &=
gt; URL:</p><p class=3DMsoNormal>>> https://github.com/kriszyp/node-l=
mdb/commit/6df903907f5516320e9a8afce45b</p><p class=3DMsoNormal>>> d3=
2ab4e8e1f2.patch</p><p class=3DMsoNormal>>> > Submission from: (NU=
LL) (71.199.6.148)</p><p class=3DMsoNormal>>> ></p><p class=3DMsoN=
ormal>>><o:p> </o:p></p><p class=3DMsoNormal>>> Thanks for=
the report and patch, added to mdb.RE/0.9</p><p class=3DMsoNormal>>>=
<o:p> </o:p></p><p class=3DMsoNormal>>> > Calling mdb_env_set=
_mapsize to increase the map size, when a DB is also</p><p class=3DMsoNorma=
l>>> in use</p><p class=3DMsoNormal>>> > by other processes,=
when MDB_WRITEMAP is enabled (and the db file has</p><p class=3DMsoNormal>=
>> been</p><p class=3DMsoNormal>>> > opened with PAGE_READWR=
ITE access), on Windows, will occasionally</p><p class=3DMsoNormal>>>=
> (seems</p><p class=3DMsoNormal>>> like</p><p class=3DMsoNormal>=
>> > about 1/100 attempts fails) produce an error "The reques=
ted operation</p><p class=3DMsoNormal>>> cannot be</p><p class=3DMsoN=
ormal>>> > performed on a file with a user-mapped section open&quo=
t;, or segfaults. The</p><p class=3DMsoNormal>>> error</p><p class=3D=
MsoNormal>>> > occurs in the SetFilePointer (or SetEndOfFile) call=
in mdb_env_map that</p><p class=3DMsoNormal>>> is</p><p class=3DMsoN=
ormal>>> > performed to increase the allocated file size to the ma=
p size, prior to</p><p class=3DMsoNormal>>> > CreateFileMapping.</=
p><p class=3DMsoNormal>>> ></p><p class=3DMsoNormal>>> > =
As it turns out this is pretty easy to solve, because manually</p><p class=
=3DMsoNormal>>> > expanding</p><p class=3DMsoNormal>>> the</=
p><p class=3DMsoNormal>>> > file size is not necessary when callin=
g CreateFileMapping with</p><p class=3DMsoNormal>>> PAGE_READWRITE</p=
><p class=3DMsoNormal>>> > access, as Windows will automatically e=
xpand the file size for us, when</p><p class=3DMsoNormal>>> opened</p=
><p class=3DMsoNormal>>> > with the page write access enabled. Of =
course, this means all processes</p><p class=3DMsoNormal>>> must be</=
p><p class=3DMsoNormal>>> > consistent in use of MDB_WRITEMAP, but=
the documentation already makes</p><p class=3DMsoNormal>>> this</p><=
p class=3DMsoNormal>>> > explicit and clear.</p><p class=3DMsoNorm=
al>>> ></p><p class=3DMsoNormal>>> > I believe this can b=
e fixed by simply adding a check for MDB_WRITEMAP</p><p class=3DMsoNormal>&=
gt;> > in</p><p class=3DMsoNormal>>> the if</p><p class=3DMsoNo=
rmal>>> > statement that calls SetFilePointer:</p><p class=3DMsoNo=
rmal>>> ></p><p class=3DMsoNormal>>> >=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!(fl=
ags & MDB_WRITEMAP) &&</p><p class=3DMsoNormal>>> >=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0 (SetFilePointer(env->me_fd,</p><p class=3DMsoNormal>>> size=
lo, &sizehi, 0)</p><p class=3DMsoNormal>>> > !=3D (DWORD)sizel=
o</p><p class=3DMsoNormal>>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 || !SetEndOfFile(env->me_fd)</p><p class=3DMsoNorm=
al>>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
|| SetFilePointer(env->me_fd, 0, NULL, 0) !=3D 0))</p><p class=3DMsoNor=
mal>>> >=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
return ErrCode();</p><p class=3DMsoNormal>>> ></p><p class=3DMsoN=
ormal>>> > The attached URL has the change as a patch/diff as appl=
ied to our node</p><p class=3DMsoNormal>>> package.</p><p class=3DMso=
Normal>>> ></p><p class=3DMsoNormal>>> > I am certainly h=
appy to just keep this change on our own branches.</p><p class=3DMsoNormal>=
>> > There</p><p class=3DMsoNormal>>> may be</p><p class=3DM=
soNormal>>> > nuances of this that I might not be aware of, but it=
seems to be</p><p class=3DMsoNormal>>> > working</p><p class=3DMs=
oNormal>>> great</p><p class=3DMsoNormal>>> > for us and I h=
ave tested this with MDB_WRITEMAP enabled and disabled.</p><p class=3DMsoNo=
rmal>>> > So</p><p class=3DMsoNormal>>> I</p><p class=3DMsoN=
ormal>>> > thought I would offer/suggest this change, as it seems =
like it is</p><p class=3DMsoNormal>>> > straightforward change to =
improve stability. Thank you!</p><p class=3DMsoNormal>>> ></p><p c=
lass=3DMsoNormal>>> ></p><p class=3DMsoNormal>>><o:p> <=
/o:p></p><p class=3DMsoNormal>>><o:p> </o:p></p><p class=3DMsoNo=
rmal>>> --</p><p class=3DMsoNormal>>>=C2=A0=C2=A0 -- Howard Chu=
</p><p class=3DMsoNormal>>>=C2=A0=C2=A0 CTO, Symas Corp.=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://www.symas.com</p><p=
class=3DMsoNormal>>>=C2=A0=C2=A0 Director, Highland Sun=C2=A0=C2=A0=
=C2=A0=C2=A0 http://highlandsun.com/hyc/</p><p class=3DMsoNormal>>>=
=C2=A0=C2=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/projec=
t/</p><p class=3DMsoNormal>>><o:p> </o:p></p><p class=3DMsoNorma=
l>><o:p> </o:p></p><p class=3DMsoNormal>> --0000000000000dacee05=
826a34a9</p><p class=3DMsoNormal>> Content-Type: text/html; charset=3D&q=
uot;UTF-8"</p><p class=3DMsoNormal>> Content-Transfer-Encoding: quo=
ted-printable</p><p class=3DMsoNormal>><o:p> </o:p></p><p class=3DM=
soNormal>> <div dir=3D3D"ltr"><div dir=3D3D"ltr&=
quot;>Thank you, I appreciate it! Do you know</p><p class=3DMsoNormal>&g=
t; h=3D ow often or when this will get into the github mirror <a</p><p c=
lass=3DMsoNormal>> href=3D3D"https://gi=3D</p><p class=3DMsoNormal>=
> thub.com/LMDB/lmdb/commits/mdb.RE/0.9">https://github.com/LMDB=
/lmdb/commi</p><p class=3DMsoNormal>> ts=3D /mdb.RE/0.9</a> (or if=
I am misunderstanding how the branches are</p><p class=3DMsoNormal>> sy=
nced)?<=3D /div></div><br><div class=3D3D"gmail_q=
uote"><div dir=3D3D"ltr"</p><p class=3DMsoNormal>> =
class=3D3D"gmail_a=3D ttr">On Sun, Feb 10, 2019 at 3:14 AM How=
ard Chu &lt;<a</p><p class=3DMsoNormal>> href=3D3D"mailto:hy=
c@s=3D ymas.com">hyc@symas.com</a>&gt;</p><p class=3DMsoN=
ormal>> wrote:<br></div><blockquote class=3D3D"gmail=
_=3D quote" style=3D3D"margin:0px</p><p class=3DMsoNormal>> 0p=
x 0px 0.8ex;border-left:1px solid rgb(204,204,=3D</p><p class=3DMsoNormal>&=
gt; 204);padding-left:1ex"><a href=3D3D"mailto:kriszyp@gmai=
l.com"</p><p class=3DMsoNormal>> target=3D3D"_blan=3D k"&=
gt;kriszyp@gmail.com</a> wrote:<br></p><p class=3DMsoNormal>>=
; &gt; Full_Name: Kristopher William Zyp<br></p><p class=3DMsoNor=
mal>> &gt; Version: LMDB 0.9.23<br></p><p class=3DMsoNormal>&g=
t; &gt; OS: Windows Server 2012 R2, Windows 10<br></p><p class=3D=
MsoNormal>> &gt; URL: <a</p><p class=3DMsoNormal>> href=3D3D&q=
uot;https://github.com/kriszyp/node-lmdb/commit/6df903907f=3D</p><p class=
=3DMsoNormal>> 5516320e9a8afce45bd32ab4e8e1f2.patch" rel=3D3D"=
noreferrer"</p><p class=3DMsoNormal>> target=3D3D"_blank"=
>=3D</p><p class=3DMsoNormal>> https://github.com/kriszyp/node-lmdb/c=
ommit/6df903907f5516320e9a8afce45bd</p><p class=3DMsoNormal>> 32=3D ab4e=
8e1f2.patch</a><br></p><p class=3DMsoNormal>> &gt; Submi=
ssion from: (NULL) (71.199.6.148)<br></p><p class=3DMsoNormal>> &a=
mp;gt; <br></p><p class=3DMsoNormal>> <br></p><p class=3DMso=
Normal>> Thanks for the report and patch, added to mdb.RE/0.9<br><=
/p><p class=3DMsoNormal>> <br></p><p class=3DMsoNormal>> &g=
t; Calling mdb_env_set_mapsize to increase the map size, when a DB is</p><p=
class=3DMsoNormal>> als=3D o in use<br></p><p class=3DMsoNormal>&=
gt; &gt; by other processes, when MDB_WRITEMAP is enabled (and the db f=
ile</p><p class=3DMsoNormal>> has =3D been<br></p><p class=3DMsoNo=
rmal>> &gt; opened with PAGE_READWRITE access), on Windows, will occ=
asionally</p><p class=3DMsoNormal>> (see=3D ms like<br></p><p clas=
s=3DMsoNormal>> &gt; about 1/100 attempts fails) produce an error &a=
mp;quot;The requested</p><p class=3DMsoNormal>> opera=3D tion cannot be&=
lt;br></p><p class=3DMsoNormal>> &gt; performed on a file with a =
user-mapped section open&quot;, or</p><p class=3DMsoNormal>> segfaul=
t=3D s. The error<br></p><p class=3DMsoNormal>> &gt; occurs in=
the SetFilePointer (or SetEndOfFile) call in mdb_env_map</p><p class=3DMso=
Normal>> tha=3D t is<br></p><p class=3DMsoNormal>> &gt; per=
formed to increase the allocated file size to the map size, prior</p><p cla=
ss=3DMsoNormal>> t=3D o<br></p><p class=3DMsoNormal>> &gt; =
CreateFileMapping.<br></p><p class=3DMsoNormal>> &gt; <br&g=
t;</p><p class=3DMsoNormal>> &gt; As it turns out this is pretty eas=
y to solve, because manually</p><p class=3DMsoNormal>> expandi=3D ng the=
<br></p><p class=3DMsoNormal>> &gt; file size is not necessary=
when calling CreateFileMapping with</p><p class=3DMsoNormal>> PAGE_RE=
=3D ADWRITE<br></p><p class=3DMsoNormal>> &gt; access, as Wind=
ows will automatically expand the file size for us,</p><p class=3DMsoNormal=
>> whe=3D n opened<br></p><p class=3DMsoNormal>> &gt; with =
the page write access enabled. Of course, this means all</p><p class=3DMsoN=
ormal>> processe=3D s must be<br></p><p class=3DMsoNormal>> &am=
p;gt; consistent in use of MDB_WRITEMAP, but the documentation already</p><=
p class=3DMsoNormal>> makes=3D=C2=A0 this<br></p><p class=3DMsoNor=
mal>> &gt; explicit and clear.<br></p><p class=3DMsoNormal>>=
; &gt; <br></p><p class=3DMsoNormal>> &gt; I believe this =
can be fixed by simply adding a check for</p><p class=3DMsoNormal>> MDB_=
WRITEMAP =3D in the if<br></p><p class=3DMsoNormal>> &gt; stat=
ement that calls SetFilePointer:<br></p><p class=3DMsoNormal>> &am=
p;gt; <br></p><p class=3DMsoNormal>> &gt;=3DC2=3DA0 =3DC2=3DA0=
=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0if (!(fla=
gs</p><p class=3DMsoNormal>> &amp=3D ; MDB_WRITEMAP) &amp;&a=
mp; (SetFilePointer(env-&gt;me_fd, sizelo,</p><p class=3DMsoNormal>>=
&amp;size=3D hi, 0)<br></p><p class=3DMsoNormal>> &gt; !=
=3D3D (DWORD)sizelo<br></p><p class=3DMsoNormal>> &gt;=3DC2=3D=
A0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=
=3DA0 =3DC2=3DA0 =3DC2=3DA0</p><p class=3DMsoNormal>> =3D =3DC2=3DA0 =3D=
C2=3DA0|| !SetEndOfFile(env-&gt;me_fd)<br></p><p class=3DMsoNorma=
l>> &gt;=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=
=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0</p><p class=3DMsoNormal>&=
gt; =3D =3DC2=3DA0 =3DC2=3DA0|| SetFilePointer(env-&gt;me_fd, 0, NULL, =
0) !=3D3D 0))<br></p><p class=3DMsoNormal>> &gt;=3DC2=3DA0 =3D=
C2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
=3DC2=3DA0 =3DC2=3DA0</p><p class=3DMsoNormal>> =3D =3DC2=3DA0 =3DC2=3DA=
0return ErrCode();<br></p><p class=3DMsoNormal>> &gt; <br&g=
t;</p><p class=3DMsoNormal>> &gt; The attached URL has the change as=
a patch/diff as applied to our</p><p class=3DMsoNormal>> node=3D=C2=A0 =
package.<br></p><p class=3DMsoNormal>> &gt; <br></p><p c=
lass=3DMsoNormal>> &gt; I am certainly happy to just keep this chang=
e on our own branches.</p><p class=3DMsoNormal>> The=3D re may be<br&=
gt;</p><p class=3DMsoNormal>> &gt; nuances of this that I might not =
be aware of, but it seems to be</p><p class=3DMsoNormal>> worki=3D ng gr=
eat<br></p><p class=3DMsoNormal>> &gt; for us and I have teste=
d this with MDB_WRITEMAP enabled and</p><p class=3DMsoNormal>> disabled.=
=3D So I<br></p><p class=3DMsoNormal>> &gt; thought I would o=
ffer/suggest this change, as it seems like it is<br></p><p class=3DMs=
oNormal>> &gt; straightforward change to improve stability. Thank yo=
u!<br></p><p class=3DMsoNormal>> &gt; <br></p><p class=
=3DMsoNormal>> &gt; <br></p><p class=3DMsoNormal>> <br&g=
t;</p><p class=3DMsoNormal>> <br></p><p class=3DMsoNormal>> -- =
<br></p><p class=3DMsoNormal>> =3DC2=3DA0 -- Howard Chu<br><=
/p><p class=3DMsoNormal>> =3DC2=3DA0 CTO, Symas Corp.=3DC2=3DA0 =3DC2=3D=
A0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a</p><p class=3DMsoNormal=
>> href=3D3D"=3D http://www.symas.com" rel=3D3D"noreferre=
r"</p><p class=3DMsoNormal>> target=3D3D"_blank">http:=
//www.symas=3D .com</a><br></p><p class=3DMsoNormal>> =3DC2=
=3DA0 Director, Highland Sun=3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0<a</p><p cla=
ss=3DMsoNormal>> href=3D3D"http://highland=3D sun.com/hyc/" re=
l=3D3D"noreferrer"</p><p class=3DMsoNormal>> target=3D3D"=
_blank">http://highlandsun.com/h=3D yc/</a><br></p><p c=
lass=3DMsoNormal>> =3DC2=3DA0 Chief Architect, OpenLDAP=3DC2=3DA0 <a<=
/p><p class=3DMsoNormal>> href=3D3D"http://www.openldap.org/p=3D ro=
ject/" rel=3D3D"noreferrer"</p><p class=3DMsoNormal>> tar=
get=3D3D"_blank">http://www.openldap.org/proje=3D ct/</a>=
;<br></p><p class=3DMsoNormal>> </blockquote></div></p=
><p class=3DMsoNormal>><o:p> </o:p></p><p class=3DMsoNormal>> --=
0000000000000dacee05826a34a9--</p><p class=3DMsoNormal>><o:p> </o:p=
></p><p class=3DMsoNormal>><o:p> </o:p></p><p class=3DMsoNormal>>=
;<o:p> </o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p class=3D=
MsoNormal><o:p> </o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p=
class=3DMsoNormal>--</p><p class=3DMsoNormal><o:p> </o:p></p><p class=
=3DMsoNormal>Quanah Gibson-Mount</p><p class=3DMsoNormal>Product Architect<=
/p><p class=3DMsoNormal>Symas Corporation</p><p class=3DMsoNormal>Packaged,=
certified, and supported LDAP solutions powered by OpenLDAP:</p><p class=
=3DMsoNormal><http://www.symas.com></p><p class=3DMsoNormal><o:p>&nbs=
p;</o:p></p><p class=3DMsoNormal><o:p> </o:p></p></div></body></html>=
--_36998F79-0E7E-483F-8FFD-6401D532832A_--