[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: LMDB mmap usage

Maybe what I was looking for was resources like this
https://0xax.gitbooks.io/linux-insides/content.

On Sat, Feb 6, 2016 at 7:54 PM, Kristoffer Sjögren <stoffe@gmail.com> wrote:
> Ok, so the page cache, that's exactly what I want to read more about.
> Not for tuning, but just to understand how the page cache works with
> the different subsystems in order to dimension the hardware properly.
> For example what IO devices affect the page cache? Why are memory
> pages reshuffled between different NUMA nodes? etc..
>
> On Sat, Feb 6, 2016 at 7:15 PM, Jay Booth <jaybooth@gmail.com> wrote:
>> Mmapped memory goes through the exact same page cache as all other
>> filesystem data -- you probably want to do exactly nothing for optimal
>> performance, the kernel will be smart about what it keeps paged in
>>
>> On Sat, Feb 6, 2016 at 12:44 PM, Kristoffer Sjögren <stoffe@gmail.com>
>> wrote:
>>>
>>> I'm not looking for fairies :-) I was just curious exactly how mmap
>>> interacts with the OS in order to understand performance metrics
>>> better.
>>>
>>> On Sat, Feb 6, 2016 at 5:46 PM, Howard Chu <hyc@symas.com> wrote:
>>> > Kristoffer Sjögren wrote:
>>> >>
>>> >> Thanks Howard. What's the best documentation on mmap you know, aside
>>> >> from read kernel source code?
>>> >
>>> >
>>> > Not sure what you're asking. Read the mmap(2) manpage and its related
>>> > pages.
>>> > If you're thinking there's something fancy hidden in there for you to
>>> > take
>>> > advantage of, I don't think you should be using mmap in the first place.
>>> >
>>> >
>>> >> On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu <hyc@symas.com> wrote:
>>> >>>
>>> >>> Kristoffer Sjögren wrote:
>>> >>>>
>>> >>>>
>>> >>>> Hi
>>> >>>>
>>> >>>> Our application do lots of caching using vmtouch, up to a point where
>>> >>>> there isn't a lot of memory left on the machine. We would like to use
>>> >>>> LMDB on the same machine to store around 40GiB data of a few hundred
>>> >>>> million entries.
>>> >>>>
>>> >>>> How can we best understand the interaction and behavior of the OS
>>> >>>> cache and sharing of memory between processes? Is LMDB doing
>>> >>>> something
>>> >>>> to help the OS?
>>> >>>
>>> >>>
>>> >>>
>>> >>> Nope, LMDB does nothing special with its mmap.
>>> >>>
>>> >>> --
>>> >>>    -- Howard Chu
>>> >>>    CTO, Symas Corp.           http://www.symas.com
>>> >>>    Director, Highland Sun     http://highlandsun.com/hyc/
>>> >>>    Chief Architect, OpenLDAP  http://www.openldap.org/project/
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> >   -- Howard Chu
>>> >   CTO, Symas Corp.           http://www.symas.com
>>> >   Director, Highland Sun     http://highlandsun.com/hyc/
>>> >   Chief Architect, OpenLDAP  http://www.openldap.org/project/
>>>
>>