[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Building for windows. Again.

To: Howard Chu <hyc@symas.com>
Subject: Re: Building for windows. Again.
From: Kristoffer Sjögren <stoffe@gmail.com>
Date: Fri, 20 Nov 2015 12:40:34 +0100
Cc: openldap-technical@openldap.org
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=0nxT0MdV9b/i47aj+qkIerL2TUZL4ElvZZEExqSy0KM=; b=bX5Q/b0pjaTkrgapl/9A8L+PLMwO93twY/W0t9kNYVPPwa/yDUrVRlnZ9oqKOboNy8 aZQYrkwoFJH0+5wrMVGyNPLTwiEyBJGp+SWNzP0EWea7HrMOM++fWwV3Iw3uqQI3OHQX a3ZkU1giHpaCEc9qbKJwmCesaOOHOnhEswpdADQLb1YXZ0BNL520/QdnSBEzePxne/Sa +BEcdvaBbrjD6wpV9bezd4/8qM9GcYRiYvi/npZTYpES70LZr3byPE7HmHKqmbIbBZjm Eno/w2d7SEZ56qtyeQsjyOwcKJULJ+tmSVhQxwDmQKz2Uuliu8b1cTpC7p8GkbFnxJly PKsw==
In-reply-to: <564F036E.4050105@symas.com>
References: <CADSBNAijT-p_cum1btC5kXTPUb2v0LkuL2N2K96iTDTJxEJJwA@mail.gmail.com> <564E37CA.9030509@symas.com> <CADSBNAiAVahKvGW_QbGDQ_L3qH9Sx69duCNFkh6qU4Te7EQ3xQ@mail.gmail.com> <CADSBNAhu6ZDwcEyh7sdxtUBgmU62Yp=9yzM+fzwMLnWgi8PJ2Q@mail.gmail.com> <CADSBNAj0BC08YF8GH+uASbE1k3JTVutbwZum4Zjy_XxdHtE75w@mail.gmail.com> <564E3FDC.4090707@symas.com> <CADSBNAgCAvo2bzhPkEmUXoq5n7rDs_PBPc=BupFQwtk_U+nzuQ@mail.gmail.com> <CADSBNAhPc6gsSJo_aRuHe_Tmb0fxpBPwkmTVK-hxvb5rRQKzPg@mail.gmail.com> <CADSBNAhS-S2XXbGO9do3if0NbjA6crBWL5vcWFY56kfJ-tPPPw@mail.gmail.com> <564E73E9.8060502@symas.com> <CADSBNAg3YaEXup59nSGwsRG+KAS6aAeyPJkJ=mYNvt3-DXkUHQ@mail.gmail.com> <564EE729.7020708@symas.com> <CADSBNAgFo6+ko5Orc-GJKDWuE7B4wFETUNyrEp5J_9BOf4ZExQ@mail.gmail.com> <CADSBNAhXLaPEx1UiJempTbVuCUa=nMjvHHnPa_mcTrgZp3ebnQ@mail.gmail.com> <564F036E.4050105@symas.com>

Nice! I'll add the fix to that release then.

On Fri, Nov 20, 2015 at 12:26 PM, Howard Chu <hyc@symas.com> wrote:
> Kristoffer Sjögren wrote:
>>
>> Just to let you know. I merged the fix manually onto LMDB_0.9.16 and it
>> works.
>
>
> Great. But as we announced just last week
> http://www.openldap.org/lists/openldap-technical/201511/msg00092.html
>
> LMDB 0.9.17 is in preparation for release now. Probably will be out in a few
> days.
>
>>
>>
>> https://github.com/deephacks/lmdbjni/commit/54a6ed3f1e0896a29e4bd10594bf9c80b0b25c65
>>
>>
>> On Fri, Nov 20, 2015 at 10:34 AM, Kristoffer Sjögren <stoffe@gmail.com>
>> wrote:
>>>
>>> The last build I know worked for lmdbjni on windows used LMDB_0.9.14.
>>>
>>> I was going to try a patch release on LMDB_0.9.16 but the fix doesn't
>>> merge cleanly.
>>>
>>> On Fri, Nov 20, 2015 at 10:26 AM, Howard Chu <hyc@symas.com> wrote:
>>>>
>>>> Kristoffer Sjögren wrote:
>>>>>
>>>>>
>>>>> Yep, the commit fixes the problem. Thanks Howard!
>>>>
>>>>
>>>>
>>>> The only question now - what is "modified" in the Java UTF-8 encoding?
>>>>
>>>>>
>>>>> On Fri, Nov 20, 2015 at 2:14 AM, Howard Chu <hyc@symas.com> wrote:
>>>>>>
>>>>>>
>>>>>> Kristoffer Sjögren wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Changing CharacterSet in the build file doesn't affect the garbled
>>>>>>> path.
>>>>>>>
>>>>>>> I tried to use explicit GetStringChars from the JNI code but with
>>>>>>> same
>>>>>>> result.
>>>>>>>
>>>>>>> Any pointers? I'm running short of ideas..
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Looks like we need to use UTF-8, as in ITS#7992. I've just merged that
>>>>>> patch
>>>>>> into mdb.master, see if that works better. You should use
>>>>>> GetStringUTFChars.
>>>>>>
>>>>>>>
>>>>>>> On Thu, Nov 19, 2015 at 11:55 PM, Kristoffer Sjögren
>>>>>>> <stoffe@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Oh, but the build file [2] have a CharacterSet element set to
>>>>>>>> Unicode.
>>>>>>>> Let me see if I can change this.
>>>>>>>>
>>>>>>>> [2]
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/deephacks/lmdbjni/blob/master/lmdbjni/src/main/native-package/vs2010.vcxproj
>>>>>>>>
>>>>>>>> On Thu, Nov 19, 2015 at 11:45 PM, Kristoffer Sjögren
>>>>>>>> <stoffe@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I can see the problem now from a local test.
>>>>>>>>>
>>>>>>>>> Hmm i'm a bit confused. The JNI code uses GetStringUTFChars calls
>>>>>>>>> [1]
>>>>>>>>> for all char * arguments going from Java to C, which is an array of
>>>>>>>>> bytes representing the string in modified UTF-8 encoding. Is this
>>>>>>>>> OK?
>>>>>>>>> The other option is to use GetStringChars which is unicode...
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/functions.html
>>>>>>>>>
>>>>>>>>> On Thu, Nov 19, 2015 at 10:32 PM, Howard Chu <hyc@symas.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kristoffer Sjögren wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The py-lmdb project have a comment [1] about reusing Python.h
>>>>>>>>>>> headers
>>>>>>>>>>> which seems to work for them.
>>>>>>>>>>>
>>>>>>>>>>> # Microsoft Visual Studio 9 ships with neither inttypes.h,
>>>>>>>>>>> stdint.h,
>>>>>>>>>>> or a
>>>>>>>>>>> sane
>>>>>>>>>>> # definition for ssize_t, so here we add lib/win32 to the search
>>>>>>>>>>> path,
>>>>>>>>>>> which
>>>>>>>>>>> # contains emulation header files provided by a third party. We
>>>>>>>>>>> force-include
>>>>>>>>>>> # Python.h everywhere since it has a portable definition of
>>>>>>>>>>> ssize_t,
>>>>>>>>>>> which
>>>>>>>>>>> # inttypes.h and stdint.h lack, and to avoid having to modify the
>>>>>>>>>>> LMDB
>>>>>>>>>>> source
>>>>>>>>>>> # code. Advapi32 is needed for LMDB's use of Windows security
>>>>>>>>>>> APIs.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Note in ITS#8067 we added an ssize_t typedef for MSVC. Also in
>>>>>>>>>> #8069
>>>>>>>>>> we
>>>>>>>>>> changed to explicitly use the ANSI APIs, so if you're using
>>>>>>>>>> Unicode
>>>>>>>>>> file
>>>>>>>>>> paths that will certainly be garbled now.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/dw/py-lmdb/blob/master/setup.py
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 19, 2015 at 10:13 PM, Kristoffer Sjögren
>>>>>>>>>>> <stoffe@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The actual build command:
>>>>>>>>>>>>
>>>>>>>>>>>> $ cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release"
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Nov 19, 2015 at 10:11 PM, Kristoffer Sjögren
>>>>>>>>>>>> <stoffe@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> That's the thing, the build doesn't complain about anything
>>>>>>>>>>>>> missing.
>>>>>>>>>>>>> But the binary seems broken because tests fail randomly.
>>>>>>>>>>>>>
>>>>>>>>>>>>> A user tried the generated binary on his machine and it worked
>>>>>>>>>>>>> but
>>>>>>>>>>>>> also
>>>>>>>>>>>>> said...
>>>>>>>>>>>>>
>>>>>>>>>>>>> "Hm, almost, the database file is no longer put to the
>>>>>>>>>>>>> requested
>>>>>>>>>>>>> directory, and its name is totally garbled. It finds it
>>>>>>>>>>>>> consistently
>>>>>>>>>>>>> though, and the contents are correct."
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/deephacks/lmdbjni/issues/48
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Nov 19, 2015 at 9:57 PM, Howard Chu <hyc@symas.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kristoffer Sjögren wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm trying to build LMDB with Java/JNI bindings with Visual
>>>>>>>>>>>>>>> C++
>>>>>>>>>>>>>>> Project Builder 9.00.30729 (vcbuild).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Unfortunately, vcbuild don't ship with inttypes.h, stdint.h,
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> sane
>>>>>>>>>>>>>>> ssize_t. So I searched around and found a few candidates [1]
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> inttypes.h and stdint.h that seems to be working for py-lmdb.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> However, unistd.h seems broken for windows - and the python
>>>>>>>>>>>>>>> guys
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>> python.h instead.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any ideas where I might find a good unistd.h?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I see you have one already - what is wrong with it, what
>>>>>>>>>>>>>> definitions
>>>>>>>>>>>>>> are
>>>>>>>>>>>>>> missing?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> -Kristoffer
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/deephacks/lmdbjni/tree/master/lmdbjni-win64/headers
>>>>>>>>>>>>>>>
>>>>
>
>
> --
>   -- Howard Chu
>   CTO, Symas Corp.           http://www.symas.com
>   Director, Highland Sun     http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP  http://www.openldap.org/project/

References:
- Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Howard Chu <hyc@symas.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Howard Chu <hyc@symas.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Howard Chu <hyc@symas.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Howard Chu <hyc@symas.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Kristoffer Sjögren <stoffe@gmail.com>
- Re: Building for windows. Again.
  - From: Howard Chu <hyc@symas.com>

Prev by Date: Getting around the single-threaded syncrepl model?
Next by Date: Tuning number of entries sent during syncrepl?
Index(es):
- Chronological
- Thread