Re: (ITS#3267) more modrdn problems

[hyc@symas.com]

entryIDs are not "recycled" unless they have fully wrapped around. Since 
entryIDs are an unsigned long, which would generally be 32 bits (or 64 
if you're on a 64bit architecture) that means you cycled through over 4 
billion add/delete operations. It doesn't sound to me like that's what 
you're doing.

The other case for holes in the entryID sequence arises because your 
input LDIF file is not well-formed. An entry's parent must already exist 
before the entry itself can be added. It is certainly possible that hole 
management is broken, but as you suggested, this code has been heavily 
used in back-bdb for over two years. back-hdb probably has not had the 
same volume of testing as back-bdb, but much of the code is identical 
anyway.

If you can supply an LDIF file (and slapd.conf) that reproduces the 
problem, that will be most useful.

ccontavalli@commedia.it wrote:

>ok, finally had some time to take a look at it...
>  probably the problem has _nothing_ to do with modrdn,
>sorry for the wrong information.
>
>Enabling debug with slapadd:
>
>=> hdb_tool_entry_put: id2entry_add failed: DB_KEYEXIST: 
>	Key/data pair already exists (-30996)
>=> hdb_tool_entry_put: txn_aborted! DB_KEYEXIST: 
>	Key/data pair already exists (-30996)
>
>by looking to bdb_id2entry_add, it seems like the generated
>e->e_id is not always uniq. Probably the bug lies somewhere into
>bdb_next_id. I did some minor playing with those functions, by
>adding some debugging messages to print the e_id out on console.
>
>  The typical situation is like the following:
>
>[...]
>added: "xxxx" (000013c1)
>added: "xxxx" (0000140f)
>=> hdb_tool_entry_put: id2entry_add failed: DB_KEYEXIST: Key/data pair
>   already exists (-30996 - 000013c4)
>=> hdb_tool_entry_put: txn_aborted! DB_KEYEXIST: Key/data pair already
>   exists (-30996)
>slapadd: could not add entry dn="xxxxx" (line=85981): txn_aborted! 
>  DB_KEYEXIST: Key/data pair already exists (-30996) (ffffffff)
>added: "xxxx" (00001410)
>added: "xxxx" (00001411)
>[...]
>
>(the xxxx represent dns I cannot disclose).
>Note that e_id is 000013c4. If I backward look in the output, something
>like:
>
>[...]
>added: "xxxx" (000013c0)
>added: "xxxx" (000013c2)
>added: "xxxx" (000013c4)
>added: "xxxx" (000013c5)
>added: "xxxx" (000013c6)
>[...]
>
>shows up. As you can see, there's a hole at 000013c3. bdb_next_id, 
>probably has some sort of ``off-by-one'' when recycling id from 
>leftover holes, and returns the value incremented by one, which 
>is already in use. This same pattern repeats for all the entries
>which slapcat fails to insert. (why holes are left at all?)
>
>Note also that some other entries fail to insert due to dn2id_add
>failing. Still need to track that down, but they could just be
>two aspects of the same problem.
>
>  This is quite a serious bug, since slapadd is either unusable,
>or if the -c option is used some ``random'' entries are not 
>entered into the database, even if a warning is printed.
>
>  but ... hey ... am I the only one using the HDB backend?? I 
>believe a bug like this should already have been noticed a long 
>ago... I'm also trying to generate a .ldiff file that shows the 
>given problem without disclosing any information regarding our 
>users...
>
>Cheers,
>Carlo
>  
>  
>


-- 
  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support