Re: Why Use accesslog overlay?

[Pete Giesin <pgiesin@hubcitymedia.com>]

Turning off the accesslog completely did increase writes but only to ~200 per sec. That is still not an acceptable number. So I need to figure out how to tweak OpenLDAP to increase write performance for both my primary backend and the accesslog backend.

Here is my configuration for the accesslog database. These are basically straight out of the box. I am not sure how best to tweak these to increase writes. My current configuration has both the db and log files for both the log and primary databases on the same disk. I realize that split these across multiple disks would help with the write performance.

Any other suggestions on how to imrpove write performance would be greatly appreciated.

Thanks,
Pete

slapd.conf
-------------------------
#Access Log database
database        hdb
suffix          "cn=log"
rootdn "cn=Manager,cn=log"
rootpw secret
directory /opt/symas/var/openldap-data/access
# Indices to maintain
index reqStart eq
index objectClass eq

DB_CONFIG
------------------------
# one 0.25 GB cache
set_cachesize 0 268435456 1

# Data Directory
#set_data_dir db

# Transaction Log settings
set_lg_regionmax 262144
set_lg_bsize 2097152

set_lg_dir /opt/symas/var/openldap-logs/access

On Fri, Mar 20, 2009 at 3:56 PM, Russell Jackson <raj@csub.edu> wrote:

Pete Giesin wrote:
> I am working on setting up OpenLDAP for a web project. I had originally planned on
> using the accesslog overlay to track all access to the LDAP. I have discovered that the
> use of this overlay has a tremendous impact on performance.
>
> I started out by auditing all operations. With this configuration I was getting 40-50
> reads/sec max. I then decided that I only needed to audit writes. Making this change
> sent the reads through the roof, but the writes were still only averaging 16-20 per
> second. I am in the process of retesting the writes with no accesslog. My assumption is
> that the writes per second will jump dramtically.
>
> So I have to question what is the purpose of the accesslog overlay? Is it really needed
> and if so is there are way to increase its performance. I have to admit that I have not
> paid attention to the settings for the accesslog backend. Do I need to tweak these
> settings just like I did for my primary backend? If so what are the optimal settings
> for a write intensive database?
>

I'm sure there will be disagreement over what I'm about to say, but if your application is
write intensive, OpenLDAP is probably the wrong tool for the job. That said, if you tune
the bdb backend for the access log correctly, you should be able to get many order of
magnitudes better performance out of it.

--
Russell A. Jackson <raj@csub.edu>
Network Analyst
California State University, Bakersfield

What the scientists have in their briefcases is terrifying.
               -- Nikita Khruschev