[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: (ITS#9017) Improving performance of commit sync in Windows
--000000000000ce7c35059edab753
Content-Type: text/plain; charset="UTF-8"
Yes, I will work on upgrading this patch to 1.0.
However, I believe in order to realize optimal Windows performance without
regression/slow-downs, I will also need to disable the use of NTDLL (and
use direct CreateFileMapping memory maps instead), as mentioned
http://www.openldap.org/lists/openldap-bugs/201902/msg00009.html. You had
mentioned in that thread that the NTDLL code wasn't destined for release,
is that still the case (it is still in the mdb.master branch)? If you are
intending to release (1.0) with the NTDLL-based memory maps (which is
understandable, can certainly be an easier path for Windows users getting
started), I will also need to add the compiler option to disable it.
Anyway, I will work on creating a PR with both these patches.
Thanks,
Kris
On Mon, Feb 17, 2020 at 8:14 AM Howard Chu <hyc@symas.com> wrote:
> Kris Zyp wrote:
> > Sorry to keep pestering, but just pinging about this patch again, as I
> still think this fix could benefit windows users. And at this point, I
> think I can say we
> > have tested it pretty well, running on our servers for almost a year :).
>
> Looks like this patch is against the 0.9 release branch. I hit a bunch of
> conflicts
> trying to apply it to mdb.master. We'll be stopping work on 0.9 soon, and
> getting
> LMDB 1.0 out the door finally, so can you please verify that your changes
> will work
> on mdb.master as well?
>
> > Thanks,
> > Kris
> >
> > On Wed, Sep 18, 2019 at 12:56 PM Kris Zyp <kriszyp@gmail.com <mailto:
> kriszyp@gmail.com>> wrote:
> >
> > Checking on this again, is this still a possibility for merging into
> LMDB? This fix is still working great (improved performance) on our systems.
> > Thanks,
> > Kris
> >
> > On Mon, Jun 17, 2019 at 1:04 PM Kris Zyp <kriszyp@gmail.com <mailto:
> kriszyp@gmail.com>> wrote:
> >
> > Is this still being considered/reviewed? Let me know if there
> are any other changes you would like me to make. This patch has continued
> to yield
> > significant and reliable performance improvements for us, and
> seems like it would be nice for this to be available for other Windows
> users.
> >
> > On Fri, May 3, 2019 at 3:52 PM Kris Zyp <kriszyp@gmail.com
> <mailto:kriszyp@gmail.com>> wrote:
> >
> > For the sake of putting this in the email thread (other code
> discussion in GitHub), here is the latest squashed commit of the proposed
> patch (with
> > the on-demand, retained overlapped array to reduce re-malloc
> and opening event handles):
> >
> https://github.com/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee222072b990f
> >
>
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>
--000000000000ce7c35059edab753
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Yes, I will work on upgrading this patch to 1.0.<div><br><=
/div><div>However, I believe in order to realize optimal Windows performanc=
e without regression/slow-downs, I will also need to disable the use of NTD=
LL (and use direct CreateFileMapping memory maps instead), as mentioned=C2=
=A0
<a href=3D"http://www.openldap.org/lists/openldap-bugs/201902/msg00009.html=
">http://www.openldap.org/lists/openldap-bugs/201902/msg00009.html</a>. You=
had mentioned in that thread that the NTDLL code wasn't destined for r=
elease, is that still the case (it is still in the mdb.master branch)? If y=
ou are intending to release (1.0) with the NTDLL-based memory maps (which i=
s understandable, can certainly be an easier path for Windows users getting=
started), I will also need to add the compiler option to disable it. Anywa=
y, I will work on creating a PR with both these patches.</div><div><br></di=
v><div>Thanks,</div><div>Kris</div></div><br><div class=3D"gmail_quote"><di=
v dir=3D"ltr" class=3D"gmail_attr">On Mon, Feb 17, 2020 at 8:14 AM Howard C=
hu <<a href=3D"mailto:hyc@symas.com";>hyc@symas.com</a>> wrote:<br></d=
iv><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord=
er-left:1px solid rgb(204,204,204);padding-left:1ex">Kris Zyp wrote:<br>
> Sorry to keep pestering, but just pinging about this patch again, as I=
still think this fix could benefit windows users. And at this point, I thi=
nk I can say we<br>
> have tested it pretty well, running on our servers for almost a year :=
).<br>
<br>
Looks like this patch is against the 0.9 release branch. I hit a bunch of c=
onflicts<br>
trying to apply it to mdb.master. We'll be stopping work on 0.9 soon, a=
nd getting<br>
LMDB 1.0 out the door finally, so can you please verify that your changes w=
ill work<br>
on mdb.master as well?<br>
<br>
> Thanks,<br>
> Kris<br>
> <br>
> On Wed, Sep 18, 2019 at 12:56 PM Kris Zyp <<a href=3D"mailto:kriszy=
p@gmail.com" target=3D"_blank">kriszyp@gmail.com</a> <mailto:<a href=3D"=
mailto:kriszyp@gmail.com"; target=3D"_blank">kriszyp@gmail.com</a>>> w=
rote:<br>
> <br>
>=C2=A0 =C2=A0 =C2=A0Checking on this again, is this still a possibility=
for merging into LMDB? This fix is still working great (improved performan=
ce) on our systems.<br>
>=C2=A0 =C2=A0 =C2=A0Thanks,<br>
>=C2=A0 =C2=A0 =C2=A0Kris<br>
> <br>
>=C2=A0 =C2=A0 =C2=A0On Mon, Jun 17, 2019 at 1:04 PM Kris Zyp <<a hre=
f=3D"mailto:kriszyp@gmail.com"; target=3D"_blank">kriszyp@gmail.com</a> <=
mailto:<a href=3D"mailto:kriszyp@gmail.com"; target=3D"_blank">kriszyp@gmail=
.com</a>>> wrote:<br>
> <br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Is this still being considered/review=
ed? Let me know if there are any other changes you would like me to make. T=
his patch has continued to yield<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0significant and reliable performance =
improvements for us, and seems like it would be nice for this to be availab=
le for other Windows users.<br>
> <br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On Fri, May 3, 2019 at 3:52 PM Kris Z=
yp <<a href=3D"mailto:kriszyp@gmail.com"; target=3D"_blank">kriszyp@gmail=
.com</a> <mailto:<a href=3D"mailto:kriszyp@gmail.com"; target=3D"_blank">=
kriszyp@gmail.com</a>>> wrote:<br>
> <br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0For the sake of putting=
this in the email thread (other code discussion in GitHub), here is the la=
test squashed commit of the proposed patch (with<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the on-demand, retained=
overlapped array to reduce re-malloc and opening event handles):<br>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"https://gith=
ub.com/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee222072b990f" r=
el=3D"noreferrer" target=3D"_blank">https://github.com/kriszyp/node-lmdb/co=
mmit/726a9156662c703bf3d453aab75ee222072b990f</a><br>
> <br>
<br>
<br>
-- <br>
=C2=A0 -- Howard Chu<br>
=C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<a href=3D"=
http://www.symas.com"; rel=3D"noreferrer" target=3D"_blank">http://www.symas=
.com</a><br>
=C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0<a href=3D"http://highland=
sun.com/hyc/" rel=3D"noreferrer" target=3D"_blank">http://highlandsun.com/h=
yc/</a><br>
=C2=A0 Chief Architect, OpenLDAP=C2=A0 <a href=3D"http://www.openldap.org/p=
roject/" rel=3D"noreferrer" target=3D"_blank">http://www.openldap.org/proje=
ct/</a><br>
</blockquote></div>
--000000000000ce7c35059edab753--