Re: LMDB use of sparse or non-sparse data file

[Geoff Swan <gswan3@bigpond.net.au>]

On 26/03/2015 8:22 AM, Geoff Swan wrote:
>
> On 26/03/2015 7:20 AM, Quanah Gibson-Mount wrote:
>> --On Wednesday, March 25, 2015 5:11 PM +1100 Geoff Swan
>> <gswan3@bigpond.net.au> wrote:
>>
>>>> Question is irrelevant since both are sparse files.
>>> Thanks for the clarification Howard.
>>> The data file from the mdb_copy snapshot was transferred over the
>>> network to the other server using scp, which I understand does not
>>> recognise sparse files, so the copy is likely to be non-sparse. I guess
>>> this would be considered a corruption and best to start with a fresh
>>> copy?
>> I've had no issue scp'ing files created from mdb_copy.
> That is interesting.
> It's good to know that there is no problem with that.
> I'll try rebuilding the db file from slapcat/slapadd.
> I don't know why an scp'd mdb_copy file did not work as intended.
The search stepped off a cliff, going from around 60s to over 10 hours.
So I rebuilt the database using slapcat/slapadd and checked that the
file is now reporting the correct sizes
569754436 -rw------- 1 ldap ldap 2000000000000 Mar 29 17:26 data.mdb

However after restarting the ldap server and performing the same search,
it is taking a huge time (it has not completed yet, after several
hours). However the same search on an identical server is returning the
results in under 2 seconds.
Checked that slapd.conf is identical, and that the modifyTimestamp is
indexed (which is what is being searched on).
Is there a way to check that the indexes are OK? It is almost as if this
index is not present, or has exceeded some internal buffer limit?


>
>> Original host:
>>
>> zimbra@zre-ldap002:~/data/ldap/mdb/db$ ls -l
>> total 432
>> -rw------- 1 zimbra zimbra 85899345920 Mar 25 14:53 data.mdb
>> -rw------- 1 zimbra zimbra        8192 Mar 25 14:54 lock.mdb
>> zimbra@zre-ldap002:~/data/ldap/mdb/db$ mkdir /tmp/mdb/
>> zimbra@zre-ldap002:~/data/ldap/mdb/db$ mdb_copy . /tmp/mdb
>> zimbra@zre-ldap002:~/data/ldap/mdb/db$ ls -l /tmp/mdb/
>> total 428
>> -rw-r--r-- 1 zimbra zimbra 438272 Mar 25 14:54 data.mdb
>>
>> Destination host:
>>
>> [build@zre-ldap003
>> zcs-NETWORK-8.7.0_BETA1_1149.RHEL6_64.20150324181030]$ scp
>> build@zre-ldap002:/tmp/mdb/data.mdb /tmp/mdb.002/
>> data.mdb 100%  428KB 428.0KB/s   00:00
>> [build@zre-ldap003
>> zcs-NETWORK-8.7.0_BETA1_1149.RHEL6_64.20150324181030]$ su
>> Password:
>> [root@zre-ldap003
>> zcs-NETWORK-8.7.0_BETA1_1149.RHEL6_64.20150324181030]# chmod a+rx
>> /tmp/mdb.002
>> [root@zre-ldap003
>> zcs-NETWORK-8.7.0_BETA1_1149.RHEL6_64.20150324181030]# chmod a+r
>> /tmp/mdb.002/data.mdb
>> [root@zre-ldap003
>> zcs-NETWORK-8.7.0_BETA1_1149.RHEL6_64.20150324181030]# su - zimbra
>> [zimbra@zre-ldap003 ~]$ cd data/ldap/mdb
>> [zimbra@zre-ldap003 mdb]$ ls
>> db.orig
>> [zimbra@zre-ldap003 mdb]$ mkdir db
>> [zimbra@zre-ldap003 mdb]$ ls
>> db  db.orig
>> [zimbra@zre-ldap003 mdb]$ cd db
>> [zimbra@zre-ldap003 db]$ cp /tmp/mdb.002/data.mdb .
>> [zimbra@zre-ldap003 db]$ ls -l
>> total 428
>> -rw-r-----. 1 zimbra zimbra 438272 Mar 25 14:57 data.mdb
>> [zimbra@zre-ldap003 db]$ ldap start
>> Started slapd: pid 32518
>> [zimbra@zre-ldap003 db]$ ls -l
>> total 432
>> -rw-r-----. 1 zimbra zimbra 85899345920 Mar 25 14:57 data.mdb
>> -rw-------. 1 zimbra zimbra        8192 Mar 25 14:57 lock.mdb
>>
>>
>>
>> --Quanah
>>
>>
>>
>> -- 
>>
>> Quanah Gibson-Mount
>> Platform Architect
>> Zimbra, Inc.
>> --------------------
>> Zimbra ::  the leader in open source messaging and collaboration
>