I have upgraded to 2.4.44, ran: # db_recover -v -c -h /var/lib/openldap/accesslog Finding last valid log LSN: file: 5 offset 4694358 Recovery starting from [4][28] Recovery complete at Mon May 16 15:12:49 2016 Maximum transaction ID 8000329e Recovery checkpoint [5][4694980] Then attempted to start and received the following: 573a1bb7 bdb_db_open: database "dc=example,dc=com": unclean shutdown detected; attempting recovery. 573a1bb7 bdb_db_open: database "cn=accesslog": unclean shutdown detected; attempting recovery. 573a1bb7 slapd starting slapd: search.c:1125: oc_filter: Assertion `f != ((void *)0)' failed. Aborted Same error(s) as before. John D. Borresen (Dave) Ph: (781) 981-1609 Email: john.borresen@ll.mit.edu -----Original Message----- From: Quanah Gibson-Mount [mailto:quanah@zimbra.com] Sent: Monday, May 16, 2016 1:47 PM To: Borresen, John - 0444 - MITLL; openldap-technical@openldap.org Subject: RE: SLAPD WON'T START ON ONE OF THE MULTIMASTERS Actually, never mind, it is not specific to mdb, I was thinking of the wrong thing. You'll have to build OpenLDAP 2.4.44 + that fix to recover. --Quanah --On Monday, May 16, 2016 6:21 PM +0000 "Borresen, John - 0444 - MITLL" <John.Borresen@ll.mit.edu> wrote: > I've run db_recover on ldapserver2, to no avail: > ># db_recover -v -h /var/lib/openldap/accesslog Finding last valid log >LSN: file: 5 offset 4659816 Recovery starting from [5][4655494] >Recovery complete at Mon May 16 13:03:19 2016 Maximum transaction ID >8000000c Recovery checkpoint [5][4660438] > ># db_recover -v -h /var/lib/openldap/openldap-data Finding last valid >log LSN: file: 3 offset 7169862 Recovery starting from [3][7169734] >Recovery complete at Mon May 16 13:03:33 2016 Maximum transaction ID >800035d9 Recovery checkpoint [3][7169862] > > Same error: ># /usr/local/openldap/libexec/slapd -u ldap -h >ldap://ldapserver2.example.come -F >/usr/local/openldap/etc/openldap/slapd.d -d 256 > 5739fe11 @(#) $OpenLDAP: slapd 2.4.40 (Sep 30 2014 16:49:45) $ > > clement@localhost.localdomain:/home/clement/build/BUILD/openldap-2.4.4 > 0/s > erv ers/slapd > 5739fe11 bdb_db_open: database "dc=group42,dc=ldap": unclean shutdown > detected; attempting recovery. > 5739fe11 bdb_db_open: database "cn=accesslog": unclean shutdown > detected; attempting recovery. > 5739fe11 slapd starting > slapd: search.c:1125: oc_filter: Assertion `f != ((void *)0)' failed. > Aborted > > I found references to remove *.bdb in the accesslog and openldap-data > directories and the __db.*. Could I run a slapcat of the dbase on > ldapserver1 and copy that over to ldapserver2? Is that a viable option? > > > > John D. Borresen (Dave) > Ph: (781) 981-1609 > Email: john.borresen@ll.mit.edu > > > -----Original Message----- > From: Quanah Gibson-Mount [mailto:quanah@zimbra.com] > Sent: Monday, May 16, 2016 11:55 AM > To: Borresen, John - 0444 - MITLL; openldap-technical@openldap.org > Subject: Re: SLAPD WON'T START ON ONE OF THE MULTIMASTERS > > That was specific to back-mdb. Your logs showed corruption with BDB. > Are you using mdb, bdb, or both? > > --Quanah > > --On Monday, May 16, 2016 3:22 PM +0000 "Borresen, John - 0444 - MITLL" > <John.Borresen@ll.mit.edu> wrote: > >> >> >> I've noticed this error/warning that keeps standing out when starting >> slapd on ldapserver2: >> >> >> >> slapd: search.c:1125: oc_filter: Assertion `f != ((void *)0)' failed. >> >> >> >> In my google searches I found this post from Quanah as a possible bug >> in >> 2.4.44 (we're running 2.4.40): >> >> >> >> >> >> This list is for discussing reported issues in OpenLDAP Software () >> >> headers >> >> quanah | 27 Apr 16:56 2016 >> >> (ITS#8413) Assertion in back-mdb/search.c during replication >> >> >> >> Full_Name: Quanah Gibson-Mount >> >> Version: 2.4.44 >> >> OS: Linux >> >> URL: ftp://ftp.openldap.org/incoming/ >> >> Submission from: (NULL) (75.111.52.177) >> >> >> >> During replication from the accesslog DB, in a 4-way MMR setup, >> various masters >> >> periodically crash with slapd: search.c:1246: oc_filter: Assertion `f >> != ((void >> >> *)0)' failed. >> >> >> >> This is back-mdb/search.c, not slapd-search.c >> >> >> >> This is triggered when a NULL filter is passed through. However, it >> should be >> >> impossible for the filter generated by str2filter to ever fail. >> >> >> >> Permalink | Reply | >> >> Navigate >> >> Go to gmane.network.openldap.bugs. >> >> Topic >> >> Go to the topic. >> >> Advertisement >> >> Project Web Page >> >> This list is for discussing reported issues in OpenLDAP Software () >> >> Search Archive >> >> >> >> Language >> >> Change language >> >> Options >> >> Current view: Threads only / Showing whole messages / Not hiding >> cited text. >> >> Change to All messages, shortened messages, or hide cited text. >> >> >> >> Post a message >> >> NNTP Newsgroup >> >> Classic Gmane web interface >> >> XML RSS Feed >> >> List Information >> >> >> >> About Gmane >> >> >> >> Gmane >> >> >> >> Again, if anyone has any suggestions as to a workaround or a >> resolution that would be most appreciative. >> >> >> >> Thanks, >> >> >> >> John D. Borresen (Dave) >> >> Email: john.borresen@ll.mit.edu >> >> >> >> >> From: openldap-technical >> [mailto:openldap-technical-bounces@openldap.org] >> On Behalf Of Borresen, John - 0444 - MITLL >> Sent: Friday, May 13, 2016 11:13 AM >> To: openldap-technical@openldap.org >> Subject: SLAPD WON'T START ON ONE OF THE MULTIMASTERS >> >> >> >> We have a 3-way multimaster configuration running on CentOS 5.11, >> OpenLDAP 2.4.40. All three have been up for years, until the other day: >> >> >> >> Slapd is running on two of the three (server names: ldapserver1, >> ldapserver2, and ldapserver3). Slapd stopped and won't restart on >> ldapserver2. >> >> >> >> From Logs on ldapserver2: >> >> May 10 04:02:13 gp42-admin4 slapd[4541]: slapd shutdown: waiting for >> 0 operations/tasks to finish >> >> May 10 04:02:19 gp42-admin4 slapd[15633]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> May 10 04:02:19 gp42-admin4 slapd[15633]: nss-ldap: do_open: >> do_start_tls >> failed:stat=-1 >> >> May 10 04:02:19 gp42-admin4 slapd[15633]: nss_ldap: reconnected to >> LDAP server ldap://ldapserver1.example.com >> >> May 10 04:02:21 gp42-admin4 slapd[15634]: bdb_db_open: database >> "cn=accesslog": database already in use. >> >> May 10 04:02:21 gp42-admin4 slapd[15634]: backend_startup_one >> (type=bdb, >> suffix="cn=accesslog"): bi_db_open failed! (-1) >> >> May 10 04:02:21 gp42-admin4 slapd[15634]: slapd stopped. >> >> May 10 04:02:22 gp42-admin4 slapd[4541]: slapd stopped. >> >> >> >> When attempting to restart slapd on server2: >> >> May 13 10:13:54 gp42-admin4 slapd[12085]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> May 13 10:13:54 gp42-admin4 slapd[12085]: nss-ldap: do_open: >> do_start_tls >> failed:stat=-1 >> >> May 13 10:13:54 gp42-admin4 slapd[12085]: nss_ldap: reconnected to >> LDAP server ldap://ldapserver1.example.com >> >> May 13 10:13:56 gp42-admin4 slapd[12086]: slapd starting >> >> May 13 10:13:56 gp42-admin4 slapd[12086]: do_syncrep2: rid=002 (4096) >> Content Sync Refresh Required >> >> May 13 10:13:56 gp42-admin4 slapd[12086]: do_syncrep2: rid=001 (4096) >> Content Sync Refresh Required >> >> May 13 10:13:57 gp42-admin4 slapd[12086]: => bdb_idl_insert_key: >> c_put id >> failed: DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock >> (-30995) >> >> May 13 10:13:57 gp42-admin4 slapd[12086]: => bdb_dn2id_add 0xfc6: >> parent >> (cn=accesslog) insert failed: -30995 >> >> May 13 10:13:57 gp42-admin4 slapd[12086]: => bdb_idl_delete_key: >> c_del id >> failed: DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock >> (-30995) >> >> May 13 10:13:57 gp42-admin4 slapd[12086]: => bdb_dn2id_delete 0xf50: >> parent (cn=accesslog) delete failed: -30995 >> >> May 13 10:15:55 gp42-admin4 slapd[12106]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> May 13 10:15:55 gp42-admin4 slapd[12106]: nss-ldap: do_open: >> do_start_tls >> failed:stat=-1 >> >> May 13 10:15:55 gp42-admin4 slapd[12106]: nss_ldap: reconnected to >> LDAP server ldap://ldapserver1.example.com >> >> May 13 10:15:55 gp42-admin4 slapd[12106]: bdb_db_open: database >> "dc=example,dc=ldap": unclean shutdown detected; attempting recovery. >> >> May 13 10:15:57 gp42-admin4 slapd[12106]: bdb_db_open: database >> "cn=accesslog": unclean shutdown detected; attempting recovery. >> >> May 13 10:15:58 gp42-admin4 slapd[12106]: slapd starting >> >> May 13 10:28:49 gp42-admin4 slapd[12255]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> May 13 10:28:49 gp42-admin4 slapd[12255]: nss-ldap: do_open: >> do_start_tls >> failed:stat=-1 >> >> May 13 10:28:49 gp42-admin4 slapd[12255]: nss_ldap: reconnected to >> LDAP server ldap://ldapserver1.example.com >> >> May 13 10:28:50 gp42-admin4 slapd[12255]: bdb_db_open: database >> "dc=example,dc=com": unclean shutdown detected; attempting recovery. >> >> May 13 10:28:50 gp42-admin4 slapd[12255]: bdb_db_open: database >> "cn=accesslog": unclean shutdown detected; attempting recovery. >> >> May 13 10:28:52 gp42-admin4 slapd[12255]: slapd starting >> >> May 13 10:29:24 gp42-admin4 slapd[12264]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> May 13 10:29:24 gp42-admin4 slapd[12264]: nss-ldap: do_open: >> do_start_tls >> failed:stat=-1 >> >> May 13 10:29:24 gp42-admin4 slapd[12264]: nss_ldap: reconnected to >> LDAP server ldap://ldapserver1.example.com >> >> May 13 10:29:24 gp42-admin4 slapd[12264]: bdb_db_open: database >> "dc=example,dc=ldap": unclean shutdown detected; attempting recovery. >> >> May 13 10:29:24 gp42-admin4 slapd[12264]: bdb_db_open: database >> "cn=accesslog": unclean shutdown detected; attempting recovery. >> >> May 13 10:29:24 gp42-admin4 slapd[12264]: slapd starting >> >> May 13 10:29:53 gp42-admin4 slapd[12280]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> May 13 10:29:53 gp42-admin4 slapd[12280]: nss-ldap: do_open: >> do_start_tls >> failed:stat=-1 >> >> May 13 10:29:53 gp42-admin4 slapd[12280]: nss_ldap: reconnected to >> LDAP server ldap://ldapserver1.example.com >> >> May 13 10:29:53 gp42-admin4 slapd[12280]: bdb_db_open: database >> "dc=example,dc=ldap": unclean shutdown detected; attempting recovery. >> >> May 13 10:29:53 gp42-admin4 slapd[12280]: bdb_db_open: database >> "cn=accesslog": unclean shutdown detected; attempting recovery. >> >> May 13 10:29:53 gp42-admin4 slapd[12280]: slapd starting >> >> May 13 10:32:35 gp42-admin4 slapd[12345]: @(#) $OpenLDAP: slapd >> 2.4.40 (Sep 30 2014 16:49:45) >> $#012#011clement@localhost.localdomain:/home/clement/build/BUILD/open >> l >> dap >> -2.4.40/servers/slapd >> >> >> >> Attempting to restart slapd from the command-line: >> >> 5735ed50 slapd starting >> >> 5735ed50 => bdb_entry_get: ndn: "cn=accesslog" >> >> 5735ed50 => bdb_entry_get: oc: "(null)", at: "(null)" >> >> 5735ed50 bdb_idl_fetch_key: %cn=accesslog >> >> 5735ed50 bdb_idl_fetch_key: [b49d1940] >> >> 5735ed50 bdb_idl_fetch_key: >> >> 5735ed50 send_ldap_result: err=0 matched="" text="" >> >> 5735ed50 => bdb_entry_get: ndn: "dc=example,dc=com" >> >> 5735ed50 => bdb_entry_get: oc: "(null)", at: "contextCSN" >> >> ldap_build_search_req ATTRS: reqDN reqType reqMod reqNewRDN >> reqDeleteOldRDN reqNewSuperior entryCSN >> >> ldap_build_search_req ATTRS: reqDN reqType reqMod reqNewRDN >> reqDeleteOldRDN reqNewSuperior entryCSN >> >> => ldap_bv2dn(uid=jdoe,ou=Users,dc=example,dc=com,0) >> >> <= ldap_bv2dn(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> => ldap_dn2bv(272) >> >> <= ldap_dn2bv(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> => ldap_dn2bv(272) >> >> <= ldap_dn2bv(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> => ldap_bv2dn(uid=jdoe,ou=Users,dc=example,dc=com,0) >> >> <= ldap_bv2dn(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> => ldap_dn2bv(272) >> >> <= ldap_dn2bv(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> => ldap_bv2dn(uid=jdoe,ou=Users,dc=example,dc=com,0) >> >> <= ldap_bv2dn(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> => ldap_dn2bv(272) >> >> <= ldap_dn2bv(uid=jdoe,ou=Users,dc=example,dc=com)=0 >> >> 5735ed50 => bdb_entry_get: ndn: "uid=jdoe,ou=Users,dc=example,dc=com" >> >> 5735ed50 => bdb_entry_get: oc: "(null)", at: "(null)" >> >> slapd: search.c:1125: oc_filter: Assertion `f != ((void *)0)' failed. >> >> Aborted >> >> >> >> >> >> I have run db_recover on the dbase(s) on ldapserver2 but to no avail. >> >> >> >> Does anyone have any suggestions? >> >> >> >> Thank you in advance for any assistance. >> >> >> >> >> >> >> >> John D. Borresen (Dave) >> >> Linux/Unix Systems Administrator >> >> MIT Lincoln Laboratory >> >> Humanitarian Assistance and Disaster Relief (HADR) Systems >> >> 244 Wood St >> >> Lexington, MA 02420 >> >> Email: john.borresen@ll.mit.edu >> >> > > > > -- > > Quanah Gibson-Mount > Platform Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration A > division of Synacor, Inc > -- Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc
Attachment:
smime.p7s
Description: S/MIME cryptographic signature