[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: [JunkMail] Re: BDB 4.3 & DB_TXN_NOT_DURABLE



Hi John,

This sounds interesting. I'll see if I can re-create it, and file another bug related to it with sleepycat.

--Quanah

--On Thursday, December 02, 2004 10:20 AM -0600 "Fortin, John {PBG}" <John.Fortin@pepsi.com> wrote:

The other odd thing.  When I tried slapadd without DB_LOG_INMEMORY, log
files were generated as expected.  However, since I didn't want to run out
of room in the filesystem I tried to run db_checkpoint and db_archive.
This caused the following (2 separate runs):

[ldap@pbglap00012 ldap]$ /usr/local/openldap-new/sbin/slapadd  -f
slapd.conf.master_new -l master-20041202.ldif
slapadd: could not add entry dn="uid=1176416,o=people,dc=pbg,dc=com"
(line=155018): txn_aborted! DB_RUNRECOVERY: Fatal error, run database
recovery (-30977)

[root@pbglap00012 master-new]# db_checkpoint -1
db_checkpoint: log.0000000008: log file open failed: No such file or
directory
db_checkpoint: PANIC: No such file or directory
db_checkpoint: DB_ENV->log_put: 8: DB_RUNRECOVERY: Fatal error, run
database recovery
db_checkpoint: txn_checkpoint: failed to flush the buffer cache
DB_RUNRECOVERY: Fatal error, run database recovery
db_checkpoint: txn_checkpoint: DB_RUNRECOVERY: Fatal error, run database
recovery
db_checkpoint: PANIC: fatal region error detected; run recovery
db_checkpoint: dbenv->close: DB_RUNRECOVERY: Fatal error, run database
recovery

After clearing the database directory and restarting slapadd:

[ldap@pbglap00012 ldap]$ /usr/local/openldap-new/sbin/slapadd  -f
slapd.conf.master_new -l master-20041202.ldif
slapadd: could not add entry dn="uid=112617,o=people,dc=pbg,dc=com"
(line=123943): txn_aborted! DB_RUNRECOVERY: Fatal error, run database
recovery (-30977)

[root@pbglap00012 master-new]# db_archive
db_archive: log.0000000010: log file open failed: No such file or
directory db_archive: PANIC: No such file or directory
db_archive: DB_ENV->log_put: 10: DB_RUNRECOVERY: Fatal error, run database
recovery
db_archive: dbenv->close: DB_RUNRECOVERY: Fatal error, run database
recovery [root@pbglap00012 master-new]# ls logs
log.0000000001  log.0000000003  log.0000000005  log.0000000007
log.0000000009
log.0000000002  log.0000000004  log.0000000006  log.0000000008
log.0000000010




John Fortin PBG Middleware and Web Services (914) 767-7844


-----Original Message-----
From: Howard Chu [mailto:hyc@symas.com]
Sent: Thursday, December 02, 2004 10:51 AM
To: Fortin, John {PBG}
Cc: OpenLDAP Mail List
Subject: Re: [JunkMail] Re: BDB 4.3 & DB_TXN_NOT_DURABLE


This is one of the big problems with BDB 4.3 that Quanah has been referring to in a lot of his emails. For reference, it's Sleepycat bug # 11505 and we are still working with them to resolve the issue. It currently makes it impractical to perform bulk loads with BDB 4.3. I somewhat regret releasing the BDB 4.3 support in OpenLDAP 2.2.19 because it's now become apparent that this was premature. BDB 4.3 does seem to work well enough in general but this aspect of it is definitely inconvenient.

Fortin, John {PBG} wrote:

Howard Chu wrote:

This flag has been superseded by DB_LOG_INMEMORY.


Except that it doesn't work as documented; the docs says the default


in-memory log buffer size is 1MB but it behaves as if >the
default size is
zero. I.e., if you don't specify a log buffer size (DB_CONFIG
set_lg_bsize)
then this DB_LOG_INMEMORY >flag doesn't work at all. It seems
to work fine
once you have it set though. I've sent a query to Sleepycat
about this, >


whether their doc or their code is wrong...

I've changed back-bdb's fasttool option in HEAD to use the
new flag, so


beware if you use it.

Howard, what do we need to be aware of? Also, with
DB_TXN_NOT_DURABLE all I
had to do was set it and forget it during the bulk load.  With
DB_LOG_INMEMORY I get the following:

[ldap@pbglap00012 ldap]$ /usr/local/openldap-new/sbin/slapadd  -f
slapd.conf.master_new -l master-20041202.ldif
slapadd: could not add entry dn="uid=5349936,o=people,dc=pbg,dc=com"
(line=1641664): txn_aborted! DB_LOG_BUFFER_FULL: In-memory
log buffer is
full (-30993)

My DB_CONFIG is as follows:

set_cachesize   0      524288000        0
set_lg_regionmax        1048576
set_lg_max              10485760
set_lg_bsize            20485760
set_lg_dir              /var/ldapdb/master-new/logs
set_tmp_dir             /tmp
set_flags DB_TXN_NOSYNC
set_flags DB_LOG_INMEMORY



John Fortin
PBG Middleware and Web Services
(914) 767-7844







--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support




-- Quanah Gibson-Mount Product Engineer Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>