Re[2]: Openldap 2.1.3 - Berkeley BDB 4.0 database backup inconsistencies

["Peter A. Savitch" <spam4octan@highway.ru>]

Hello Tony,

Tuesday, July 30, 2002, 4:53:40 PM, you wrote:

TE> O.k., did it - though "checkpoint 256 0" to save time. This is a test
TE> installation with little relative change. Up to then I did
TE> "db_checkpoint -1" at uneven intervals.

Db_archive(1) says that You should stop DB operations prior to
db_checkpoint, if You want Your archive to be time determined.

Sorry, I was terribly wrong about the exact meaning of `checkpoint'
back-bdb(5) directive:

 *** `log checkpointing'  and `log rotation' are DIFFERENT things ***

It does exactly what it's name for, and nothing else -- just
checkpointing log, not file rotation. I tried to set checkpoint to 1k
to get the limit. Since checkpointing in slapd occurs after
pthread_yield(), this gave me 100% CPU utilization, a lot of detached
threads, and, finally, slapd crash (I've made about 20k  additions). I
guess You should not checkpoint Your log files too often, especially
during high system load. 

I'll try openldap-devel -- maybe people could tell me something
interesting. 

Next, to set the MAXIMUM LOG FILE SIZE one should call another
function: 

http://www.sleepycat.com/docs/api_c/env_set_lg_max.html

See also: http://www.sleepycat.com/docs/ref/log/limits.html

The idea is NOT to make log files too small. Well, 10M is a good
default. So there should be another back-bdb(5) option to handle it.

>> /usr/bin/db_archive -a -h /path/to/ldap/bdb | xargs gzip -9

TE> That's no good, it doesn't do anything. db_archive is no archiving tool,
TE> it just gives info. to stdout or one can pipe the result (then you need
TE> -l or -s) -. But not to gzip.

Not to gzip, but to `xargs gzip`. This runs gzip with the filenames
given in stdin to xargs.

TE> But, thanks to you, I got the courage to go further. I now see that I
TE> could write a shell script (no, not perl) backup routine which could
TE> backup and/or restore the whole database with impunity, (including
TE> splitting up, renaming and zipping old log files), BUT I can't see any
TE> way of getting rid of old log files, at the moment. When, as an
[...]
TE> This is *really badly* documented stuff. However, as sysadmin, I more
[...]

No.
To get rid of old log files You should call db_archive. Again, without
-s and -a this prints the names of `old' log files. But please, please
read db_archive(1) manpage with attention, section "DB ARCHIVAL
PROCEDURES" -- this covers all possible scenarios.
The idea is simple -- if You've got the snapshot and all of the
required log files, then You're lucky.

Actually these things appear to be *only* BDB stuff and may be
considered off-topic here. Let's go off-list.

-- 
Best regards,
 Peter                            mailto:spam4octan@highway.ru