[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: BDB database maintenance

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Jeremy Ardley

> I quote
> http://www.sleepycat.com/company/pdfs/newbdbrelationalwhite.pdf
>
>
> Data Administration May Not Be Needed
>
> Eliminating SQL often means eliminating or reducing database
> administration
> costs, too. Berkeley DB installations seldom need a DBA. In
> many Berkeley DB
> deployments, the end user may not even realize that a
> database component is
> part of the product - how many cell phone users think about a
> data table
> lookup when searching for a number in their personal address book?
>
> Berkeley DB operates without end-user backups, installation,
> or tuning.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ^^^^^^^^^
>
> My question to the developers of back-bdb is why does the
> back-bdb use of
> Berkeley DB require end-user backups and tuning?  This includes the
> requirement to mananage the log files and to perform database
> recovery after
> abnormal shutdown.
>
> Is the Berkeley article just plain wrong?  Or is there
> something special about back-bdb ?

There are countless things "special" about back-bdb. Please read the FAQ:
http://www.openldap.org/faq/data/cache/756.html

How many cell-phones do you know of with end-user selectable memory sizes,
disk capacity, and/or operating system version?

In the embedded application market, of course you can preconfigure
everything; because you only have one platform configuration to deal with.
Think about what you're asking for, do you really believe "one size fits
all"???

back-ldbm using Berkeley DB does not generate log files, but you still need
to tune the dbcachesize, and back-ldbm doesn't offer high concurrency or
transaction-protected updates. After an abnormal shutdown, you generally need
to delete the database and start over from scratch, because when it gets
corrupted there is no telling what's left inside.

back-bdb uses the BerkeleyDB API to the fullest extent, using every special
feature that BerkeleyDB provides. It is very much bleeding edge technology in
this respect, and sometimes we run into faults in BDB itself. This is why the
minimum BDB version keeps getting higher with newer OpenLDAP releases; as we
run into problems with BDB we report the issues to SleepyCat and work around
them in the meantime.

The last bit is true for all of the packages we use, by the way - I've
found/reported/fixed dozens of bugs in Cyrus SASL, Heimdal Kerberos, OpenSSL,
BDB, libtool, etc... OpenLDAP 2.1 has exercised all of these packages in ways
no one else ever has before, which tends to uncover issues those maintainers
never accounted for.

It is very easy to statically configure a system when you know that it will
only be used one way. This luxury doesn't apply to OpenLDAP. To get the
absolute best performance out of BDB with transaction logging, you need more
than one physical disk. Not everyone who uses OpenLDAP has multiple spindles
on their system. There's no way with autoconf or RPM to determine this
dynamically. To get the best caching performance, you have to allocate a lot
of RAM. There is no portable way to determine how much RAM is available on a
system. Even knowing how much physical memory is installed, there is no way
to dynamically determine how much of it can be hogged up by OpenLDAP, and how
much must be left for other users on the system. There is no way to
predetermine the number of clients that will be connected to the system, so
there is no way to preconfigure the sizes of the lock tables or any other
configurable item. All of these things depend on the particular installation,
and all of these are things that the system administrator must know and must
take into account themself.

You cannot add code to detect these parameters on the fly either, because
then you have a system that is no longer running under anyone's explicit
control. How would you like it if you installed OpenLDAP on a box and then
suddenly all your other apps fell over dead, because OpenLDAP decided to use
all of the available disk and memory space for itself?

If you believe all of this stuff should have pefectly tuned defaults out of
the box, you're just being naive.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support