[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: Re: (ITS#9017) Improving performance of commit sync in Windows
- To: openldap-its@OpenLDAP.org
- Subject: RE: Re: (ITS#9017) Improving performance of commit sync in Windows
- From: kriszyp@gmail.com
- Date: Tue, 18 Feb 2020 20:06:44 +0000
- Auto-submitted: auto-generated (OpenLDAP-ITS)
<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc=
hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of=
fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40";><head><meta ht=
tp-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta name=
=3DGenerator content=3D"Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=3DEN-CA link=3Dblue vlink=3D"#954F72"><div cla=
ss=3DWordSection1><p class=3DMsoNormal>Looking into the APIs, I would guess=
that CreateFileMapping=E2=80=99s dwMaximumSizeHigh and dwMaximumSizeLow ar=
guments are mapped to NtCreateSection=E2=80=99s MaximumSize argument (4<sup=
>th</sup> argument). In LMDB=E2=80=99s 0.9 CreateFileMapping call, the size=
is provided (since it is required), whereas NtCreateSection has MaximumSiz=
e as optional, and in LMDB (mdb.master), it is not provided. Passing the si=
ze in (with MaximumSize argument) to NtCreateSection seems to result in the=
same behavior as CreateFileMapping of preallocating a file size equivalent=
to the max/mapping size, rather than dynamically growing.</p><p class=3DMs=
oNormal><o:p> </o:p></p><p class=3DMsoNormal>It is challenging to read=
ily test for the performance issues; it took a fair bit of effort and load =
testing to trigger the long-duration performance/freezing issues we had see=
n before on Azure. However, in some more isolated tests, providing a Maximu=
mSize to NtCreateSection does seem to perform better than leaving it unspec=
ified (specifically the system CPU usage seems to about be about twice as m=
uch with dynamic growth than fixed file size), and the performance seems ab=
out the same as CreateFileMapping. And I think we both had suspected that i=
t was likely due to the overhead or inefficiencies in how Windows was dynam=
ically growing the (size of the) mapped file, and having Windows used a pre=
defined fixed file size, whether through NtCreateSection or CreateFileMappi=
ng, probably would prevent these slowdown issues.</p><p class=3DMsoNormal><=
o:p> </o:p></p><p class=3DMsoNormal>But, whether we use NtCreateSectio=
n or CreateFileMapping, we are still faced with the same trade-off of wheth=
er to use dynamic Windows-controlled file growth (convenient since the map =
size can be set arbitrarily large), or using pre-specified file size (to av=
oid the slowing/freezing that seems to sometimes result from dynamic Window=
s growth). So I assume it would still be reasonable to offer a compile time=
option, so users could opt-in to using a fixed file size (provide the Maxi=
mumSize to NtCreateSection)? Fortunately this would be a trivial addition, =
just a few lines of code to use the preprocessor switch to supply this argu=
ment.</p><p class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal><a =
href=3D"http://undocumented.ntinternals.net/index.html?page=3DUserMode%2FUn=
documented%20Functions%2FNT%20Objects%2FSection%2FNtMapViewOfSection.html">=
http://undocumented.ntinternals.net/index.html?page=3DUserMode%2FUndocument=
ed%20Functions%2FNT%20Objects%2FSection%2FNtMapViewOfSection.html</a></p><p=
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Thanks,<br>Kri=
s</p><p class=3DMsoNormal><o:p> </o:p></p><div style=3D'mso-element:pa=
ra-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm =
0cm 0cm'><p class=3DMsoNormal style=3D'border:none;padding:0cm'><b>From: </=
b><a href=3D"mailto:hyc@symas.com";>Howard Chu</a><br><b>Sent: </b>February =
18, 2020 8:15 AM<br><b>To: </b><a href=3D"mailto:kriszyp@gmail.com";>Kris Zy=
p</a><br><b>Cc: </b><a href=3D"mailto:openldap-its@openldap.org";>openldap-i=
ts@OpenLDAP.org</a><br><b>Subject: </b>Re: (ITS#9017) Improving performance=
of commit sync in Windows</p></div><p class=3DMsoNormal><o:p> </o:p><=
/p><p class=3DMsoNormal>Kris Zyp wrote:</p><p class=3DMsoNormal>> Yes, I=
will work on upgrading this patch to 1.0.</p><p class=3DMsoNormal>> </p=
><p class=3DMsoNormal>> However, I believe in order to realize optimal W=
indows performance without regression/slow-downs, I will also need to disab=
le the use of NTDLL (and use direct</p><p class=3DMsoNormal>> CreateFile=
Mapping memory maps instead), as mentioned http://www.openldap.org/li=
sts/openldap-bugs/201902/msg00009.html. You had mentioned in that thread th=
at</p><p class=3DMsoNormal>> the NTDLL code wasn't destined for release,=
is that still the case (it is still in the mdb.master branch)? If you are =
intending to release (1.0) with the</p><p class=3DMsoNormal>> NTDLL-base=
d memory maps (which is understandable, can certainly be an easier path for=
Windows users getting started), I will also need to add the compiler</p><p=
class=3DMsoNormal>> option to disable it. Anyway, I will work on creati=
ng a PR with both these patches.</p><p class=3DMsoNormal><o:p> </o:p><=
/p><p class=3DMsoNormal>Might be interesting to investigate why the behavio=
r with NTDLL is so much slower on Azure. Since</p><p class=3DMsoNormal>the =
Win32 APIs are implemented on top of the NT APIs, you would expect them to =
perform identically.</p><p class=3DMsoNormal>Perhaps there's some additiona=
l flag that the Win32 API is passing to NT that is needed on Azure.</p><p c=
lass=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>> </p><p clas=
s=3DMsoNormal>> Thanks,</p><p class=3DMsoNormal>> Kris</p><p class=3D=
MsoNormal>> </p><p class=3DMsoNormal>> On Mon, Feb 17, 2020 at 8:14 A=
M Howard Chu <hyc@symas.com <mailto:hyc@symas.com>> wrote:</p><=
p class=3DMsoNormal>> </p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=
=A0 Kris Zyp wrote:</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 &g=
t; Sorry to keep pestering, but just pinging about this patch again, as I s=
till think this fix could benefit windows users. And at this point, I think=
I can</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 say we</p><p cl=
ass=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 > have tested it pretty wel=
l, running on our servers for almost a year :).</p><p class=3DMsoNormal>>=
; </p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 Looks like this pat=
ch is against the 0.9 release branch. I hit a bunch of conflicts</p><p clas=
s=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 trying to apply it to mdb.master=
. We'll be stopping work on 0.9 soon, and getting</p><p class=3DMsoNormal>&=
gt;=C2=A0=C2=A0=C2=A0=C2=A0 LMDB 1.0 out the door finally, so can you pleas=
e verify that your changes will work</p><p class=3DMsoNormal>>=C2=A0=C2=
=A0=C2=A0=C2=A0 on mdb.master as well?</p><p class=3DMsoNormal>> </p><p =
class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 > Thanks,</p><p class=3DM=
soNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 > Kris</p><p class=3DMsoNormal>>=
;=C2=A0=C2=A0=C2=A0=C2=A0 ></p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=
=A0=C2=A0 > On Wed, Sep 18, 2019 at 12:56 PM Kris Zyp <kriszyp@gmail.=
com <mailto:kriszyp@gmail.com> <mailto:kriszyp@gmail.com <mailt=
o:kriszyp@gmail.com>>> wrote:</p><p class=3DMsoNormal>>=C2=A0=
=C2=A0=C2=A0=C2=A0 ></p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=
=A0 > Checking on this again, is this still a possibi=
lity for merging into LMDB? This fix is still working great (improved perfo=
rmance) on our systems.</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=
=A0 > Thanks,</p><p class=3DMsoNormal>>=C2=A0=C2=
=A0=C2=A0=C2=A0 > Kris</p><p class=3DMsoNormal>>=
=C2=A0=C2=A0=C2=A0=C2=A0 ></p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=
=A0=C2=A0 > On Mon, Jun 17, 2019 at 1:04 PM Kris Zyp =
<kriszyp@gmail.com <mailto:kriszyp@gmail.com> <mailto:kriszyp@g=
mail.com <mailto:kriszyp@gmail.com>>> wrote:</p><p class=3DMsoN=
ormal>>=C2=A0=C2=A0=C2=A0=C2=A0 ></p><p class=3DMsoNormal>>=C2=A0=
=C2=A0=C2=A0=C2=A0 > Is this still bein=
g considered/reviewed? Let me know if there are any other changes you would=
like me to make. This patch has continued to yield</p><p class=3DMsoNormal=
>>=C2=A0=C2=A0=C2=A0=C2=A0 > signifi=
cant and reliable performance improvements for us, and seems like it would =
be nice for this to be available for other Windows users.</p><p class=3DMso=
Normal>>=C2=A0=C2=A0=C2=A0=C2=A0 ></p><p class=3DMsoNormal>>=C2=A0=
=C2=A0=C2=A0=C2=A0 > On Fri, May 3, 201=
9 at 3:52 PM Kris Zyp <kriszyp@gmail.com <mailto:kriszyp@gmail.com>=
; <mailto:kriszyp@gmail.com <mailto:kriszyp@gmail.com>>> wro=
te:</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 ></p><p class=
=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 > &=
nbsp; For the sake of putting this in the email thread (other =
code discussion in GitHub), here is the latest squashed commit of the propo=
sed patch (with</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 >&n=
bsp; the on-demand, retained overl=
apped array to reduce re-malloc and opening event handles):</p><p class=3DM=
soNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 >  =
; https://github.com/kriszyp/node-lmdb/commit/726a9156662c703b=
f3d453aab75ee222072b990f</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=
=A0 ></p><p class=3DMsoNormal>> </p><p class=3DMsoNormal>> </p><p =
class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 -- </p><p class=3DMsoNormal>=
>=C2=A0=C2=A0=C2=A0=C2=A0 -- Howard Chu</p><p class=3DMsoNormal>&=
gt;=C2=A0=C2=A0=C2=A0=C2=A0 CTO, Symas Corp. &nb=
sp; http://www.symas.com</p><p class=3DMsoNormal>>=C2=A0=C2=
=A0=C2=A0=C2=A0 Director, Highland Sun http://hig=
hlandsun.com/hyc/</p><p class=3DMsoNormal>>=C2=A0=C2=A0=C2=A0=C2=A0 &nbs=
p; Chief Architect, OpenLDAP http://www.openldap.org/project/</p><p c=
lass=3DMsoNormal>> </p><p class=3DMsoNormal><o:p> </o:p></p><p clas=
s=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>-- </p><p class=3DM=
soNormal>=C2=A0=C2=A0-- Howard Chu</p><p class=3DMsoNormal>=C2=A0 CTO, Syma=
s Corp.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://=
www.symas.com</p><p class=3DMsoNormal>=C2=A0 Director, Highland Sun=C2=A0=
=C2=A0=C2=A0=C2=A0 http://highlandsun.com/hyc/</p><p class=3DMsoNormal>=C2=
=A0 Chief Architect, OpenLDAP=C2=A0 http://www.openldap.org/project/</p><p =
class=3DMsoNormal><o:p> </o:p></p></div></body></html>=