Re: help with mdb database recovery after crash

[Andrei Mikhailovsky <andrei@arhont.com>]

Thanks everyone for their contribution. I will follow the suggestions over the weekend and update the list on the progress.

Kind regards

----- Original Message -----
> From: "Enes Kıdık" <enes.kidik@pardus.org.tr>
> To: "Andrei Mikhailovsky" <andrei@arhont.com>
> Sent: Thursday, 7 February, 2019 10:01:00
> Subject: Re: help with mdb database recovery after crash

> Hello Andrei,
> I am not posting this to openldap maillist but It may be useful to you.
> 
> 3 years ago I was working on zimbra collaboration and had hard times with those
> sparse files.
> I found the links I read. Your situation seems different from mine but take a
> shot.
> 
> you may want to search how to copy an old backup of zimbra sparse file to new
> location (after using mdb_copy copied file is no longer a a sparse file so
> there may be another way like slapcat-slapadd)
> https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning_8.0
> https://www.openldap.org/lists/openldap-technical/201305/msg00229.html
> https://linuxacademy.com/blog/linux/openldap-fixing-or-recovering-a-corrupt-directory/
> 
> Quannah is ldap-guru. He may give you a specific answer to solve your problem.
> 
> Have a good day. I hope you solve it quickly.
> Enes
> 
> ----- Original Message -----
> From: "Howard Chu" <hyc@symas.com>
> To: "Andrei Mikhailovsky" <andrei@arhont.com>, openldap-technical@openldap.org
> Sent: Thursday, February 7, 2019 5:42:40 AM
> Subject: Re: help with mdb database recovery after crash
> 
> Andrei Mikhailovsky wrote:
>> Hello everyone,
>> 
>> I have a bit of an issue with my ldap database. I have a Zimbra community
>> edition which uses openldap. A server crashed and I am unable to start the ldap
>> services after the reboot. The description of my problem, after some digging
>> about is:
>> 
>> 
>> the initial error indicated problem with the ldap
>> 
>> Starting ldap...Done.
>> Search error: Unable to determine enabled services from ldap.
>> Enabled services read from cache. Service list may be inaccurate.
>> 
>> Having investigated the issue, I noticed the following errors in the zimbra.log
>> 
>> *slapd[31281]: mdb_entry_decode: attribute index 560427631 not recognized*
>> 
>> I also noticed that the /opt/zimbra/data/ldap/mdb/db/data.mdb is actually 81Gb
>> in size and had reached the limit imposed by the ldap_db_maxsize variable. so
>> over the weekend, the LDAP mdb file became no longer sparse.
>> 
>> I tried following the steps described in
>> https://syslint.com/blog/tutorial/solved-critical-ldap-primary-mdb-database-is-90-full-in-zimbra/
>> but with no success, as the slapcat segfaults with the following message.
>> 
>> 
>> /opt/zimbra/common/sbin/slapcat -ccc -F /opt/zimbra/data/ldap/config -b "" -l
>> /opt/zimbra/RECOVERY/SLAPCAT/zimbra_mdb.ldiff
>> 5c583982 mdb_entry_decode: attribute index 560427631 not recognized
>> Segmentation fault (core dumped)
>> 
>> the mdb_copy produces a file of 420 mb in size, but it still contains the
>> "mdb_entry_decode: attribute index 560427631 not recognized" error.
>> I've also tried mdb_dump, but had the same issues after using the mdb_load
>> command.
>> 
>> I found a post ( http://www.openldap.org/its/index.cgi/Software%20Bugs?id=8360 ;)
>> in the openldap community that the mdb gets corrupted if it reaches the maximum
>> defined size. but no solution of how to fix it.
> 
> That's from over 3 years ago and has subsequently fixed. If you're running on
> such an old
> release, there's likely not much that can be done. Ordinarily it's possible to
> back up to
> the immediately preceding transaction, in case the last transaction is
> corrupted, but with
> that particular bug it's likely that the corruption occurred in an earlier
> transaction
> and has been carried forward in all subsequent ones.
>> 
>> any advice on getting the database recovered and working again?
> 
> You could try using the preceding transaction and see if it's in any better
> shape. The code
> for this is not released in LMDB 0.9. You can compile the mdb.master branch in
> git to obtain
> it. Then use the "-v" option with mdb_copy and see if that copy of the database
> is usable.
> 
> --
>  -- Howard Chu
>  CTO, Symas Corp.           http://www.symas.com
>  Director, Highland Sun     http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP  http://www.openldap.org/project/