back-hdb exhausting free list (ITS#2698)

[lukeh@padl.com]

Full_Name: Luke Howard
Version: HEAD
OS: Linux
URL: 
Submission from: (NULL) (165.228.130.11)


This problem has existed for some months (perhaps forever), but I have never
reported it officially. __memp_alloc() spins waiting for memory to become
available, that never becomes available. In the stack trace below, you will note
that __db_shalloc() is attempting to allocate 16447 bytes; I note that when I
change BDB_ID2ENTRY_PAGESIZE, the stall occurs in the same place, always in
attempting to allocate (BDB_ID2ENTRY_PAGESIZE + 63) bytes. The entry that is
added is only 880 bytes long, so I'm not sure why it's trying to allocate so
much memory.

(gdb) bt
#0  __db_shalloc (p=0x40806000, len=16447, align=8, retp=0xbfffe200) at
../dist/../env/db_salloc.c:134
#1  0x08184ed0 in __memp_alloc (dbmp=0x831fb08, memreg=0x831fb60,
mfp=0x408477d8, len=16447, offsetp=0x0, retp=0xbfffe2dc)
    at ../dist/../mp/mp_alloc.c:108
#2  0x081984a0 in __memp_fget (dbmfp=0x8320600, pgnoaddr=0xbfffe31c, flags=4,
addrp=0xbfffe320)
    at ../dist/../mp/mp_fget.c:297
#3  0x08166690 in __db_new (dbc=0x8320908, type=5, pagepp=0xbfffe39c) at
../dist/../db/db_meta.c:159
#4  0x081446a2 in __bam_root (dbc=0x8320908, cp=0x8320a3c) at
../dist/../btree/bt_split.c:201
#5  0x0814453d in __bam_split (dbc=0x8320908, arg=0xbfffe800,
root_pgnop=0xbfffe518) at ../dist/../btree/bt_split.c:138
#6  0x0819003e in __bam_c_put (dbc=0x8320908, key=0xbfffe800, data=0xbfffe7e0,
flags=17, pgnop=0xbfffe5cc)
    at ../dist/../btree/bt_cursor.c:1978
#7  0x0815e597 in __db_c_put (dbc_arg=0x8320908, key=0xbfffe800,
data=0xbfffe7e0, flags=17) at ../dist/../db/db_cam.c:1434
#8  0x08155bf9 in __db_put (dbp=0x83202d0, txn=0x8321388, key=0xbfffe800,
data=0xbfffe7e0, flags=0)
    at ../dist/../db/db_am.c:650
#9  0x080b6265 in bdb_id2entry_put (be=0x830d990, tid=0x8321388, e=0x8326768,
flag=23) at id2entry.c:52
#10 0x080b629a in hdb_id2entry_add (be=0x830d990, tid=0x8321388, e=0x8326768) at
id2entry.c:69
#11 0x0809f47e in hdb_tool_entry_put (be=0x830d990, e=0x8326768,
text=0xbfffea28) at tools.c:330
#12 0x08064ab5 in main (argc=7, argv=0xbfffec34) at slapadd.c:245
#13 0x40184627 in __libc_start_main (main=0x8064310 <main>, argc=7,
ubp_av=0xbfffec34, init=0x806328c <_init>, 
    fini=0x81a0f40 <_fini>, rtld_fini=0x4000dcd4 <_dl_fini>,
stack_end=0xbfffec2c) at ../sysdeps/generic/libc-start.c:129

The cause is not obvious, but it is apparent that there are no suitably sized
blocks on the free list:

(gdb) call __db_shalloc_dump(p, stderr)
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Memory free list
0x40806004: 1044        0x408068a4: 4164        0x408079ac: 12500      
0x4080ab44: 12500       0x4080dcdc: 12500      
0x40810e74: 8332      0x40812fc4: 12500       0x4081615c: 12500      
0x408192ec: 12500       0x4081c47c: 212 0x40820664:
12500    0x408237f4: 8332 0x40825944: 8332        0x40827a94: 12500      
0x4082ac2c: 12500       0x4082ddc4: 12500      
0x40830f5c: 12500     0x408340f4: 12500       0x4083728c: 12500      
0x4083a424: 12500       0x4083d5bc: 8332

Note that although I am using back-hdb, I can duplicate the problem with
back-bdb; this is not surprising given where it occurs. The problem occurs both
when adding entries via slapd (ldapmodify) and via slapadd. The only
distinguishing thing about the data being imported appears to be the presence of
some very large entries; one, of course, would not expect this to be a problem.

Interestingly, if I run it under valgrind, it appears to silently abort;
id2entry.bdb and log.0000000001 are created, though, but slapcat shows nothing.

Another interesting thing to note is that, if I only load N entries first
(including the "large" entry, which is the first), and then restart slapd, the
rest of the import works fine; N seems to be about 11 but varies according to
the data.