RE: Corrupt index files

["Howard Chu" <hyc@symas.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Darren Gamble

> Good day,

> > > Do you get these problems if you
> > > use the ldbm
> > > backend in 2.0.25 with BDB?
> >
> > Yes.  I got exactly the same results with 2.0.21, 2.0.23 and
> > 2.0.25 (yes, I
> > have been trying very, very hard to get something working).

You apparently don't read documentation very well.

> I've made an interesting discovery this afternoon.  I installed the Red Hat
> 2.0.23 RPM back on the test server, but on a hunch, I tried to load the
> information back with slapadd rather than my backup of the .gdbm database
> files.

While the database formats generally don't change, this is not the documented
way to migrate databases between software upgrades. You should always dump
the
database into LDIF format before upgrading to a new version.

On little-endian (e.g., Intel) boxes, the byte-order of index keys changed
between 2.0 and 2.1, so using your gdbm backups between 2.0 and 2.1 will
fail.

Always use slapcat and slapadd for backups, that's what they're for.

> Lo and behold, using gdbm I STILL have this problem (although the entries
> that it duplicates are different than the ones with bdb) !  I've never had
> to import data in with slapadd before, so, I probably just never noticed
> this.  I tried 2.0.25 with gdbm just for kicks; same result.
>
> So, it looks like it's not the backend, but OpenLDAP itself, that
> is causing
> these import problems.  It's probably a separate problem from my original
> index problem.

Since other people have reported indexing problems with gdbm that disappeared
when switching to BDB, I am very skeptical about your conclusion. Since you
claim to never have used slapadd before, then it's clear that you have never
actually executed a slapd with BDB running in the backend, because BDB and
gdbm
have completely incompatible file formats. The only possible way to migrate
from
a gdbm installation to a BDB installation is via slapcat/slapadd.

Given the haphazard-sounding nature of your upgrade "methodology", I'd guess
that your 2.0.25 was built with gdbm and never used any BDB code.

> I guess should consider following the CVS tree in order to get a
> working set of tools.

For you, I think this would be a serious mistake.

> A few past messages on the mailing list suggest that there are
> some problematic differences in the API, though.  Should I be able to
> upgrade to 2.1.X if the rest of my LDAP tools were compiled with 2.0.X ?

The ldap* tools won't much care. The slap* tools must all match.
>
> Is it possible that it's only slapadd that has this problem?  Furthermore,
> if I want to attempt to work around this problem by recreating all of my
> index files after a slapadd, which files do I need to keep intact?  I note
> that slapindex doesn't seem to alter index files if they already exist.

Both slapadd and slapindex are additive programs. They will not overwrite
data that already exists, they will only insert new data. If you want to
recreate your indices from scratch, delete everything but the id2entry and
the dn2id databases, then run slapindex.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support