[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapcat and BDB transaction-logs





--On August 15, 2008 1:51:37 PM +0200 tamarin p <tamarinp@gmail.com> wrote:


Hello,

I'm trying to migrate a directory from OpenLDAP 2.3.x to an OpenLDAP
2.4.11 test installation. My LDAP-experience is less than extensive, but
I've run slapcat on the old base, and am now trying to import it back
into 2.4.11 with slapadd. The base is large enough that the import will
take several days at the current rate (3gb LDIF/approx 4 million
entries), and so I can't monitor the process at all times.

My problem is with the backend transaction logs, though. These are
chucked out at a rate of approximately one per minute until the disk runs
out of space. I did a quick google on the issue and found that you should
be able to remove logs not currently involved in any transactions by
running db_archive -d
(http://www.oracle.com/technology/documentation/berkeley-db/db/ref/transa
pp/logfile.html). I've also tried adding a set_flags DB_LOG_AUTOREMOVE in
my DB_CONFIG with no luck. Then I figured I could set up a cronjob to run
db_archive hourly, but when I run db_archive -d (or more to the point,
slapd_db_archive2.4 -d, as I've installed using Buchan Milne's RPMs), not
a single log-file is removed. Moreover, none of the *.bdb files are
increasing in size while slapadd is running, only additional log-files
and __db.001 etc.Does slapadd basically do all adds in one huge
transaction before committing at the end, or am I doing something wrong?
Can I simply erase log-files with rm on the fly instead? The
checkpointing in slapd.conf is set to 256k/5min as per default. Anything
else I'm missing?


What flags are you using to slapadd? What version of BDB is your openldap build linked against? It's certainly worked for me to clean up the transaction logs while slapadding.


-Quanah


--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration