[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapd stability problems with add/change operations

To: Quanah Gibson-Mount <quanah@stanford.edu>, openldap-software@OpenLDAP.org
Subject: Re: slapd stability problems with add/change operations
From: Adrian Gschwend <adrian.gschwend@bfh.ch>
Date: Fri, 19 Aug 2005 09:24:44 +0200
In-reply-to: <77B509B2566458D5B5DD2755@cadabra-dsl.stanford.edu>
Organization: Berne University of Applied Sciences
References: <42FB1C4E.9060107@bfh.ch> <6FC6D33A4DA4A4448FE19372@cadabra-dsl.stanford.edu> <42FCAC1E.7070607@bfh.ch> <153A900B9FEF5A9C5B91117B@[10.0.0.1]> <4300B213.6010601@bfh.ch> <4301F126.10100@bfh.ch> <1BDC4B9DEB87DB8DD24CC257@[10.0.0.1]> <43034228.5040201@bfh.ch> <44579ABB68BDFD699EC3AEC0@[10.0.0.1]> <43047B73.40403@bfh.ch> <77B509B2566458D5B5DD2755@cadabra-dsl.stanford.edu>
User-agent: Mozilla Thunderbird 1.0.2 (X11/20050317)

Quanah Gibson-Mount wrote:

No, it makes little sense to me. I always build with "-O2 -g" so I have an optimized build with debugging, and it functions no differently than as just "-O2".

Maybe you fixed something in your rebuild. ;)

The optimizations should be done as before. However, I think the debug version just hits the issue less, I could reproduce it again this morning and I could take the backtrace now.

I post the file as attachment, I hope this is ok like this. I am not sure if the backtrace is complete because I have an error message at the end.

Anyway, I hope it gives some more insights about what is going on here, it starts to get a bit annoying ;) BTW if it helps we could organize remote access to a box as well with some test-data (neutralized passwords for example).

Also note that I will be offline for a week now, I should be back on August, 29th.

thanks

Adrian
--
Adrian Gschwend
System Administrator
Berne University of Applied Sciences
Biel, Switzerland

ldap-01# gdb /usr/local/libexec/slapd 1566
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
Attaching to program: /usr/local/libexec/slapd, process 1566
Reading symbols from /usr/local/lib/libldap_r-2.2.so.7...done.
Loaded symbols for /usr/local/lib/libldap_r-2.2.so.7
Reading symbols from /usr/local/lib/liblber-2.2.so.7...done.
Loaded symbols for /usr/local/lib/liblber-2.2.so.7
Reading symbols from /usr/local/lib/libdb-4.2.so.2...done.
Loaded symbols for /usr/local/lib/libdb-4.2.so.2
Reading symbols from /lib/libcrypto.so.3...done.
Loaded symbols for /lib/libcrypto.so.3
Reading symbols from /usr/lib/libssl.so.3...done.
Loaded symbols for /usr/lib/libssl.so.3
Reading symbols from /usr/lib/libfetch.so.3...done.
Loaded symbols for /usr/lib/libfetch.so.3
Reading symbols from /usr/lib/libcom_err.so.2...done.
Loaded symbols for /usr/lib/libcom_err.so.2
Reading symbols from /lib/libcrypt.so.2...done.
Loaded symbols for /lib/libcrypt.so.2
Reading symbols from /usr/lib/libwrap.so.3...done.
Loaded symbols for /usr/lib/libwrap.so.3
Reading symbols from /usr/lib/libpthread.so.1...done.
Loaded symbols for /usr/lib/libpthread.so.1
Reading symbols from /lib/libc.so.5...done.
Loaded symbols for /lib/libc.so.5
Reading symbols from /libexec/ld-elf.so.1...done.
0x2814792b in ldap_pvt_thread_yield () from /usr/local/lib/libldap_r-2.2.so.7

(gdb) thr apply all bt
Thread 8 (Thread 8 (sleeping)):
#0  0x2839dc7b in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#1  0x2839dbcd in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#2  0x283917a2 in pthread_join () from /usr/lib/libpthread.so.1
#3  0x281478e6 in ldap_pvt_thread_join () from /usr/local/lib/libldap_r-2.2.so.7
#4  0x0805840d in slapd_daemon () at daemon.c:2041
#5  0x0804e622 in main (argc=7, argv=0xbfbfec4c) at main.c:713

Thread 7 (Thread 7 (LWP 100151)):
#0  0x283a5ef7 in pthread_testcancel () from /usr/lib/libpthread.so.1
#1  0x2839e39f in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#2  0x00000000 in ?? ()

Thread 6 (Thread 6 (runnable)):
#0  0x283f9dd7 in select () from /lib/libc.so.5
#1  0x28390639 in select () from /usr/lib/libpthread.so.1
#2  0x0805745e in slapd_daemon_task (ptr=0x0) at daemon.c:1422
#3  0x28392a8d in pthread_create () from /usr/lib/libpthread.so.1
#4  0x2844cc5f in _ctx_start () from /lib/libc.so.5

Thread 5 (Thread 5 (runnable)):
#0  0x2839dc7b in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#1  0x2839dbcd in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#2  0x283936ef in sched_yield () from /usr/lib/libpthread.so.1
#3  0x28147928 in ldap_pvt_thread_yield () from /usr/local/lib/libldap_r-2.2.so.7
#4  0x2821d84a in __os_yield_4002 () from /usr/local/lib/libdb-4.2.so.2
#5  0x281929c3 in __db_tas_mutex_lock_4002 () from /usr/local/lib/libdb-4.2.so.2
#6  0x2820a6f8 in __lock_get_internal () from /usr/local/lib/libdb-4.2.so.2
#7  0x2820968d in __lock_vec_4002 () from /usr/local/lib/libdb-4.2.so.2
#8  0x281f17c7 in __db_lget_4002 () from /usr/local/lib/libdb-4.2.so.2
#9  0x281a174b in __bam_search_4002 () from /usr/local/lib/libdb-4.2.so.2
#10 0x281987f0 in __bam_c_search () from /usr/local/lib/libdb-4.2.so.2
#11 0x28195afb in __bam_c_get () from /usr/local/lib/libdb-4.2.so.2
#12 0x281e6f73 in __db_c_get_4002 () from /usr/local/lib/libdb-4.2.so.2
#13 0x281ef68b in __db_c_get_pp_4002 () from /usr/local/lib/libdb-4.2.so.2
#14 0x080c2229 in bdb_idl_fetch_key (be=0x1, db=0x81aa800, tid=0x0, key=0xbf62c538, ids=0x830e000) at idl.c:519
#15 0x080c3edb in bdb_key_read (be=0x8132d00, db=0x81aa800, txn=0x0, k=0x95dbb54, ids=0x830e000) at key.c:52
#16 0x080c0574 in equality_candidates (op=0x8b3d800, ava=0x94dc114, ids=0x838e000, tmp=0x830e000) at filterindex.c:364
#17 0x080bf9a3 in bdb_filter_candidates (op=0x8b3d800, f=0x94dc13c, ids=0x838e000, tmp=0x830e000, stack=0x840e000)
    at filterindex.c:92
#18 0x080bfe0b in list_candidates (op=0x8b3d800, flist=0xbf62c6d8, ftype=160, ids=0xbf66c9d8, tmp=0x830e000, save=0x838e000)
    at filterindex.c:176
#19 0x080bfc0e in bdb_filter_candidates (op=0x8b3d800, f=0xbf62c708, ids=0xbf66c9d8, tmp=0x830e000, stack=0x838e000)
    at filterindex.c:127
#20 0x0809fb5e in search_candidates (stackop=0x8e41400, op=0x8b3d800, rs=0xbf6ecda8, e=0xbf62c9a8, locker=2147487024,
    ids=0xbf66c9d8, scopes=0xbf62c9d8) at search.c:1665
#21 0x0809dfd0 in bdb_do_search (op=0x8e41400, rs=0xbf6ecda8, sop=0x8b3d800, ps_e=0x8e4a240, ps_type=1) at search.c:746
#22 0x0809d2fa in bdb_psearch (op=0x8e41400, rs=0xbf6ecda8, sop=0x8b3d800, ps_e=0x8e4a240, ps_type=1) at search.c:392
#23 0x080b987c in bdb_add (op=0x8e41400, rs=0xbf6ecda8) at add.c:434
#24 0x0805f469 in do_add (op=0x8e41400, rs=0xbf6ecda8) at add.c:318
#25 0x08059fef in connection_operation (ctx=0xbf6ece28, arg_v=0x8e41400) at connection.c:1054
#26 0x281471dc in ldap_int_thread_pool_wrapper () from /usr/local/lib/libldap_r-2.2.so.7
#27 0x28392a8d in pthread_create () from /usr/lib/libpthread.so.1
#28 0x2844cc5f in _ctx_start () from /lib/libc.so.5

Thread 4 (Thread 4 (sleeping)):
#0  0x2839dc7b in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#1  0x2839dbcd in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#2  0x283a3701 in _pthread_cond_wait () from /usr/lib/libpthread.so.1
#3  0x283a39d2 in pthread_cond_wait () from /usr/lib/libpthread.so.1
#4  0x281479d2 in ldap_pvt_thread_cond_wait () from /usr/local/lib/libldap_r-2.2.so.7
#5  0x281466de in ldap_pvt_thread_rdwr_wlock () from /usr/local/lib/libldap_r-2.2.so.7
#6  0x0809d5cd in bdb_do_search (op=0x8e41c00, rs=0xbf2ebda8, sop=0x8e41c00, ps_e=0x0, ps_type=0) at search.c:484
#7  0x0809d2db in bdb_search (op=0x1, rs=0xbf2ebda8) at search.c:384
#8  0x0805bf88 in do_search (op=0x8e41c00, rs=0xbf2ebda8) at search.c:412
#9  0x0805a102 in connection_operation (ctx=0xbf2ebe28, arg_v=0x8e41c00) at connection.c:1079
#10 0x281471dc in ldap_int_thread_pool_wrapper () from /usr/local/lib/libldap_r-2.2.so.7
#11 0x28392a8d in pthread_create () from /usr/lib/libpthread.so.1
#12 0x2844cc5f in _ctx_start () from /lib/libc.so.5

Thread 3 (Thread 3 (runnable)):
#0  0x2839dc7b in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#1  0x2839dbcd in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#2  0x283936ef in sched_yield () from /usr/lib/libpthread.so.1
#3  0x28147928 in ldap_pvt_thread_yield () from /usr/local/lib/libldap_r-2.2.so.7
#4  0x2821d84a in __os_yield_4002 () from /usr/local/lib/libdb-4.2.so.2
#5  0x281929c3 in __db_tas_mutex_lock_4002 () from /usr/local/lib/libdb-4.2.so.2
#6  0x2820a6f8 in __lock_get_internal () from /usr/local/lib/libdb-4.2.so.2
#7  0x2820968d in __lock_vec_4002 () from /usr/local/lib/libdb-4.2.so.2
#8  0x281f17c7 in __db_lget_4002 () from /usr/local/lib/libdb-4.2.so.2
#9  0x281a174b in __bam_search_4002 () from /usr/local/lib/libdb-4.2.so.2
#10 0x281987f0 in __bam_c_search () from /usr/local/lib/libdb-4.2.so.2
#11 0x28195afb in __bam_c_get () from /usr/local/lib/libdb-4.2.so.2
#12 0x281e6f73 in __db_c_get_4002 () from /usr/local/lib/libdb-4.2.so.2
#13 0x281ef68b in __db_c_get_pp_4002 () from /usr/local/lib/libdb-4.2.so.2
#14 0x080c2229 in bdb_idl_fetch_key (be=0x1, db=0x81aa800, tid=0x0, key=0xbee2a5c8, ids=0x1523a000) at idl.c:519
#15 0x080c3edb in bdb_key_read (be=0x8132d00, db=0x81aa800, txn=0x0, k=0x1513a2ac, ids=0x1523a000) at key.c:52
#16 0x080c0574 in equality_candidates (op=0x8e4b200, ava=0xbee2a7d8, ids=0x1533a000, tmp=0x1523a000) at filterindex.c:364
#17 0x080bf9a3 in bdb_filter_candidates (op=0x8e4b200, f=0xbee2a808, ids=0x1533a000, tmp=0x1523a000, stack=0x153ba000)
    at filterindex.c:92
#18 0x080bfe0b in list_candidates (op=0x8e4b200, flist=0xbee2a808, ftype=161, ids=0x152ba000, tmp=0x1523a000,
    save=0x1533a000) at filterindex.c:176
#19 0x080bfc79 in bdb_filter_candidates (op=0x8e4b200, f=0xbee2a7f8, ids=0x152ba000, tmp=0x1523a000, stack=0x1533a000)
    at filterindex.c:133
#20 0x080bfe0b in list_candidates (op=0x8e4b200, flist=0xbee2a7e8, ftype=160, ids=0xbee6aae8, tmp=0x1523a000,
    save=0x152ba000) at filterindex.c:176
#21 0x080bfc0e in bdb_filter_candidates (op=0x8e4b200, f=0xbee2a818, ids=0xbee6aae8, tmp=0x1523a000, stack=0x152ba000)
    at filterindex.c:127
#22 0x0809fb5e in search_candidates (stackop=0x8e4b200, op=0x8e4b200, rs=0xbeeeada8, e=0xbee2aab8, locker=71,
    ids=0xbee6aae8, scopes=0xbee2aae8) at search.c:1665
#23 0x0809dfd0 in bdb_do_search (op=0x8e4b200, rs=0xbeeeada8, sop=0x8e4b200, ps_e=0x0, ps_type=0) at search.c:746
#24 0x0809d2db in bdb_search (op=0x1, rs=0xbeeeada8) at search.c:384
#25 0x0805bf88 in do_search (op=0x8e4b200, rs=0xbeeeada8) at search.c:412
#26 0x0805a102 in connection_operation (ctx=0xbeeeae28, arg_v=0x8e4b200) at connection.c:1079
#27 0x281471dc in ldap_int_thread_pool_wrapper () from /usr/local/lib/libldap_r-2.2.so.7
#28 0x28392a8d in pthread_create () from /usr/lib/libpthread.so.1
#29 0x2844cc5f in _ctx_start () from /lib/libc.so.5

Thread 2 (Thread 2 (sleeping)):
#0  0x2839dc7b in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#1  0x2839dbcd in pthread_mutexattr_init () from /usr/lib/libpthread.so.1
#2  0x283a3701 in _pthread_cond_wait () from /usr/lib/libpthread.so.1
#3  0x283a39d2 in pthread_cond_wait () from /usr/lib/libpthread.so.1
#4  0x281479d2 in ldap_pvt_thread_cond_wait () from /usr/local/lib/libldap_r-2.2.so.7
#5  0x281466de in ldap_pvt_thread_rdwr_wlock () from /usr/local/lib/libldap_r-2.2.so.7
#6  0x0809d5cd in bdb_do_search (op=0x8e4b800, rs=0xbeae9da8, sop=0x8e4b800, ps_e=0x0, ps_type=0) at search.c:484
#7  0x0809d2db in bdb_search (op=0x1, rs=0xbeae9da8) at search.c:384
#8  0x0805bf88 in do_search (op=0x8e4b800, rs=0xbeae9da8) at search.c:412
#9  0x0805a102 in connection_operation (ctx=0xbeae9e28, arg_v=0x8e4b800) at connection.c:1079
#10 0x281471dc in ldap_int_thread_pool_wrapper () from /usr/local/lib/libldap_r-2.2.so.7
#11 0x28392a8d in pthread_create () from /usr/lib/libpthread.so.1
#12 0x2844cc5f in _ctx_start () from /lib/libc.so.5

Thread 1 (Thread 1 (LWP 100376)):
#0  0x2814792b in ldap_pvt_thread_yield () from /usr/local/lib/libldap_r-2.2.so.7
#1  0x2821d84a in __os_yield_4002 () from /usr/local/lib/libdb-4.2.so.2
#2  0x281929c3 in __db_tas_mutex_lock_4002 () from /usr/local/lib/libdb-4.2.so.2
#3  0x2820a6f8 in __lock_get_internal () from /usr/local/lib/libdb-4.2.so.2
#4  0x2820968d in __lock_vec_4002 () from /usr/local/lib/libdb-4.2.so.2
#5  0x281f17c7 in __db_lget_4002 () from /usr/local/lib/libdb-4.2.so.2
#6  0x281a174b in __bam_search_4002 () from /usr/local/lib/libdb-4.2.so.2
#7  0x281987f0 in __bam_c_search () from /usr/local/lib/libdb-4.2.so.2
#8  0x28195afb in __bam_c_get () from /usr/local/lib/libdb-4.2.so.2
#9  0x281e6f73 in __db_c_get_4002 () from /usr/local/lib/libdb-4.2.so.2
#10 0x281ef68b in __db_c_get_pp_4002 () from /usr/local/lib/libdb-4.2.so.2
#11 0x080c2229 in bdb_idl_fetch_key (be=0x0, db=0x81aa800, tid=0x0, key=0xbe6285c8, ids=0x15c4e000) at idl.c:519
#12 0x080c3edb in bdb_key_read (be=0x8132d00, db=0x81aa800, txn=0x0, k=0x15b4e2ac, ids=0x15c4e000) at key.c:52
#13 0x080c0574 in equality_candidates (op=0x8e4be00, ava=0xbe6287d8, ids=0x15d4e000, tmp=0x15c4e000) at filterindex.c:364
#14 0x080bf9a3 in bdb_filter_candidates (op=0x8e4be00, f=0xbe628808, ids=0x15d4e000, tmp=0x15c4e000, stack=0x15dce000)
    at filterindex.c:92
#15 0x080bfe0b in list_candidates (op=0x8e4be00, flist=0xbe628808, ftype=161, ids=0x15cce000, tmp=0x15c4e000,
    save=0x15d4e000) at filterindex.c:176
#16 0x080bfc79 in bdb_filter_candidates (op=0x8e4be00, f=0xbe6287f8, ids=0x15cce000, tmp=0x15c4e000, stack=0x15d4e000)
    at filterindex.c:133
#17 0x080bfe0b in list_candidates (op=0x8e4be00, flist=0xbe6287e8, ftype=160, ids=0xbe668ae8, tmp=0x15c4e000,
    save=0x15cce000) at filterindex.c:176
#18 0x080bfc0e in bdb_filter_candidates (op=0x8e4be00, f=0xbe628818, ids=0xbe668ae8, tmp=0x15c4e000, stack=0x15cce000)
    at filterindex.c:127
#19 0x0809fb5e in search_candidates (stackop=0x8e4be00, op=0x8e4be00, rs=0xbe6e8da8, e=0xbe628ab8, locker=73,
    ids=0xbe668ae8, scopes=0xbe628ae8) at search.c:1665
#20 0x0809dfd0 in bdb_do_search (op=0x8e4be00, rs=0xbe6e8da8, sop=0x8e4be00, ps_e=0x0, ps_type=0) at search.c:746
#21 0x0809d2db in bdb_search (op=0x0, rs=0xbe6e8da8) at search.c:384
#22 0x0805bf88 in do_search (op=0x8e4be00, rs=0xbe6e8da8) at search.c:412
#23 0x0805a102 in connection_operation (ctx=0xbe6e8e28, arg_v=0x8e4be00) at connection.c:1079
#24 0x281471dc in ldap_int_thread_pool_wrapper () from /usr/local/lib/libldap_r-2.2.so.7
#25 0x28392a8d in pthread_create () from /usr/lib/libpthread.so.1
#26 0x2844cc5f in _ctx_start () from /lib/libc.so.5
Cannot get lwp 0 registers: Invalid argument

Follow-Ups:
- Re: slapd stability problems with add/change operations
  - From: Aaron Richton <richton@nbcs.rutgers.edu>

References:
- slapd stability problems with add/change operations
  - From: Adrian Gschwend <adrian.gschwend@bfh.ch>
- Re: slapd stability problems with add/change operations
  - From: Quanah Gibson-Mount <quanah@stanford.edu>
- Re: slapd stability problems with add/change operations
  - From: Adrian Gschwend <adrian.gschwend@bfh.ch>
- Re: slapd stability problems with add/change operations
  - From: Quanah Gibson-Mount <quanah@stanford.edu>
- Re: slapd stability problems with add/change operations
  - From: Adrian Gschwend <adrian.gschwend@bfh.ch>
- Re: slapd stability problems with add/change operations
  - From: Adrian Gschwend <adrian.gschwend@bfh.ch>
- Re: slapd stability problems with add/change operations
  - From: Quanah Gibson-Mount <quanah@stanford.edu>
- Re: slapd stability problems with add/change operations
  - From: Adrian Gschwend <adrian.gschwend@bfh.ch>
- Re: slapd stability problems with add/change operations
  - From: Quanah Gibson-Mount <quanah@stanford.edu>
- Re: slapd stability problems with add/change operations
  - From: Adrian Gschwend <adrian.gschwend@bfh.ch>
- Re: slapd stability problems with add/change operations
  - From: Quanah Gibson-Mount <quanah@stanford.edu>

Prev by Date: Re: Connections timing out to OpenLDAP server
Next by Date: Re: Connections timing out to OpenLDAP server
Index(es):
- Chronological
- Thread