Re: LMDB use of sparse or non-sparse data file

[Geoff Swan <gswan3@bigpond.net.au>]

On 24/03/2015 6:38 AM, Howard Chu wrote:
> Geoff Swan wrote:
>> I had to duplicate an LMDB database for replication recently, and
>> used mdb_copy to do so.
>> One server is using the original data.mdb database (which is sparse)
> and the other is using the mdb_copy non-sparse data.mdb file.
>
> If you specified no special options, the file produced by mdb_copy is
> identical to the original - it will also be sparse if the original is.
>
>> The two servers are identical (hardware, OS, software and
> configuration). OpenLDAP-2.4.39 is being used, 64 bit Linux OS. mdb_stat
> shows the map size as the same, which is expected.
>>
>> Will the use of the non-sparse file cause any performance issues?
>
> Question is irrelevant since both are sparse files.
Thanks for the clarification Howard.
The data file from the mdb_copy snapshot was transferred over the
network to the other server using scp, which I understand does not
recognise sparse files, so the copy is likely to be non-sparse. I guess
this would be considered a corruption and best to start with a fresh copy?

>
>> The reason for asking is that I am seeing a difference in search
>> times between the two.
>> With 20 million objects, a search on modifyTimestamp (which is
>> indexed) gives:
>> server 1: approx 1s
>> server 2: approx 60s
>>
>> server 2 started with the same search time as server 1 when the
> databases were originally copied, but has slowly increased its search
> time over about a week for this same search.
>
> Look at disk I/O and memory usage, the LMDB file itself has no bearing
> here.
>