Re: (ITS#9017) Improving performance of commit sync in Windows

[hyc@symas.com]

Kris Zyp wrote:
> Yes, I will work on upgrading this patch to 1.0.
>=20
> However, I believe in order to realize optimal Windows performance with=
out regression/slow-downs, I will also need to disable the use of NTDLL (=
and use direct
> CreateFileMapping memory maps instead), as mentioned=C2=A0 http://www.o=
penldap.org/lists/openldap-bugs/201902/msg00009.html. You had mentioned i=
n that thread that
> the NTDLL code wasn't destined for release, is that still the case (it =
is still in the mdb.master branch)? If you are intending to release (1.0)=
 with the
> NTDLL-based memory maps (which is understandable, can certainly be an e=
asier path for Windows users getting started), I will also need to add th=
e compiler
> option to disable it. Anyway, I will work on creating a PR with both th=
ese patches.

Might be interesting to investigate why the behavior with NTDLL is so muc=
h slower on Azure. Since
the Win32 APIs are implemented on top of the NT APIs, you would expect th=
em to perform identically.
Perhaps there's some additional flag that the Win32 API is passing to NT =
that is needed on Azure.

>=20
> Thanks,
> Kris
>=20
> On Mon, Feb 17, 2020 at 8:14 AM Howard Chu <hyc@symas.com <mailto:hyc@s=
ymas.com>> wrote:
>=20
>     Kris Zyp wrote:
>     > Sorry to keep pestering, but just pinging about this patch again,=
 as I still think this fix could benefit windows users. And at this point=
, I think I can
>     say we
>     > have tested it pretty well, running on our servers for almost a y=
ear :).
>=20
>     Looks like this patch is against the 0.9 release branch. I hit a bu=
nch of conflicts
>     trying to apply it to mdb.master. We'll be stopping work on 0.9 soo=
n, and getting
>     LMDB 1.0 out the door finally, so can you please verify that your c=
hanges will work
>     on mdb.master as well?
>=20
>     > Thanks,
>     > Kris
>     >
>     > On Wed, Sep 18, 2019 at 12:56 PM Kris Zyp <kriszyp@gmail.com <mai=
lto:kriszyp@gmail.com> <mailto:kriszyp@gmail.com <mailto:kriszyp@gmail.co=
m>>> wrote:
>     >
>     >=C2=A0 =C2=A0 =C2=A0Checking on this again, is this still a possib=
ility for merging into LMDB? This fix is still working great (improved pe=
rformance) on our systems.
>     >=C2=A0 =C2=A0 =C2=A0Thanks,
>     >=C2=A0 =C2=A0 =C2=A0Kris
>     >
>     >=C2=A0 =C2=A0 =C2=A0On Mon, Jun 17, 2019 at 1:04 PM Kris Zyp <kris=
zyp@gmail.com <mailto:kriszyp@gmail.com> <mailto:kriszyp@gmail.com <mailt=
o:kriszyp@gmail.com>>> wrote:
>     >
>     >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Is this still being considered/r=
eviewed? Let me know if there are any other changes you would like me to =
make. This patch has continued to yield
>     >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0significant and reliable perform=
ance improvements for us, and seems like it would be nice for this to be =
available for other Windows users.
>     >
>     >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On Fri, May 3, 2019 at 3:52 PM K=
ris Zyp <kriszyp@gmail.com <mailto:kriszyp@gmail.com> <mailto:kriszyp@gma=
il.com <mailto:kriszyp@gmail.com>>> wrote:
>     >
>     >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0For the sake of pu=
tting this in the email thread (other code discussion in GitHub), here is=
 the latest squashed commit of the proposed patch (with
>     >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the on-demand, ret=
ained overlapped array to reduce re-malloc and opening event handles):
>     >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://github.com=
/kriszyp/node-lmdb/commit/726a9156662c703bf3d453aab75ee222072b990f
>     >
>=20
>=20
>     --=20
>     =C2=A0 -- Howard Chu
>     =C2=A0 CTO, Symas Corp.=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0htt=
p://www.symas.com
>     =C2=A0 Director, Highland Sun=C2=A0 =C2=A0 =C2=A0http://highlandsun=
.com/hyc/
>     =C2=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/proj=
ect/
>=20


--=20
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/