Re: Transaction logging and OpenLDAP

["matthew sporleder" <msporleder@gmail.com>]

On 8/11/06, Tech Guy <techguymenlo@yahoo.com> wrote:
> Hi,
>
> I understand that there have been several discussions
> in the past about transaction logging on OpenLDAP. I
> have a couple of questions in this matter that may or
> may not have been addressed previously.
>
> We are running BDBv4.4.20 and OpenLDAPv2.3.24.
>
> 1. Is there a way to completely stop transaction
> logging in OpenLDAP, along the lines of "slapadd -q"?
> >From reading BDB documentation, it appears that an
> application can use BDB as a transactional or
> non-transactional back-end.
>
> 2. Presently we have the following directives to
> remove log files in slapd.conf:
> checkpoint      5120     15
>
> and in DB_CONFIG
> set_flags DB_LOG_AUTOREMOVE
>
> These directives reduce the logging in OpenLDAP but
> under stress add's, a 10M log file is creates about
> every two hours. Looking at db_stat, it appears the
> checkpoint is not done as frequently. Is this unusual
> and are we missing a setting?
>
> We now have a cron job to force a checkpoint and
> remove unwanted log files.
>
> # Commit buffered active transactions to database
> every mid-night
> * 0 * * * /usr/bin/db_checkpoint -1 -h /data/ldbm
>
> #Remove in-active log files ten minutes after
> midnight.
> 10 0 * * * /usr/bin/db_archive -d -h /data/ldbm"
>
> Looking for suggestions to avoid logging or an
> efficient way to remove logs.
>
> Thanks in advance for responses,
>
> TechGuy

* 0 * * * /usr/bin/db_checkpoint -1 -h /data/ldbm  <-- I hope that's a
typo.  It would create 60 checkpoints at the 0 hour.

I thought I remembered reading about a DB_CONFIG variable that would,
essentially, turn of transactions for bdb (thus eliminating the logs).
However, I don't think there's a good reason to do so.  My testing
has shown that ldbm (the non-transactional bdb) is simply slower.  The
I/O from these logs files is minimal, and they don't even take up much
space with autoremove turned on.

The DB_LOG_AUTOREMOVE makes the db_archive redundant, by the way.
I don't think forcing the checkpoint is necessary, either.  Especially
if you don't want to keep the log files around, and only doing one/day
doesn't make a lot of sense unless it's followed by a few other
scripts for backups.  Why not run them every hour?  Or just set the
set_lg_max in DB_CONFIG to control the size of the transaction logs,
and autoremove will take care of the ones that aren't needed for
running.  10MB of write activity for an hour isn't exactly taxing on
any modern system.

I'll note, however, that getting rid of these logs eliminates one
possible avenue of recovery.  All backups must now be taken using
ldapsearch (for live backups) or with slapcat (for downed backups).

What problem are you actually trying to solve?