[Date Prev][Date Next] [Chronological] [Thread] [Top]
Re: (ITS#6334) hang during ldapmodify

To: openldap-its@OpenLDAP.org
Subject: Re: (ITS#6334) hang during ldapmodify
From: mkd@cs.brown.edu
Date: Mon, 19 Oct 2009 16:02:42 GMT
Auto-submitted: auto-generated (OpenLDAP-ITS)
This is a multi-part message in MIME format.
--------------030103000807090505050604
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Matthew,

Attached is a first shot at the backtrace.  It looks like I have many
missing symbol tables, so I'm not sure how useful this will be to you.
The hang appears to be in pthread, so it may not strictly be a slapd
issue after all.

As a temporary measure, I'm looking at a workaround on our end that
breaks up the replace calls into adds and deletes.  This may let us work
around this entire problem.  It doesn't address the underlying issue though.

Thanks,

Mark

> Can you attach a gdb backtrace taken during the hang?  (thread apply all
> backtrace full)

--------------030103000807090505050604
Content-Type: text/plain;
 name="gdb-slapd.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="gdb-slapd.txt"

GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
(gdb) attach 14884
Attaching to process 14884
Reading symbols from /sysvol/ldap/release/2.4.19/lib/slapd...(no debugging symbols found)...done.
Reading symbols from /sysvol/ldap/release/2.4.19/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /sysvol/ldap/release/2.4.19/lib/libldap_r-2.4.so.2
Reading symbols from /sysvol/ldap/release/2.4.19/lib/liblber-2.4.so.2...done.
Loaded symbols for /sysvol/ldap/release/2.4.19/lib/liblber-2.4.so.2
Reading symbols from /usr/lib/libdb-4.6.so...done.
Loaded symbols for /usr/lib/libdb-4.6.so
Reading symbols from /usr/lib/libodbc.so.1...done.
Loaded symbols for /usr/lib/libodbc.so.1
Reading symbols from /lib/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0xb6dc79b0 (LWP 14884)]
[New Thread 0xa1143b90 (LWP 14886)]
[New Thread 0xa1543b90 (LWP 14885)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/libslp.so.1...done.
Loaded symbols for /usr/lib/libslp.so.1
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/libicuuc.so.38...done.
Loaded symbols for /usr/lib/libicuuc.so.38
Reading symbols from /usr/lib/libicudata.so.38...warning: Lowest section in /usr/lib/libicudata.so.38 is .hash at 000000b4
done.
Loaded symbols for /usr/lib/libicudata.so.38
Reading symbols from /usr/lib/libsasl2.so.2...done.
Loaded symbols for /usr/lib/libsasl2.so.2
Reading symbols from /usr/lib/libgnutls.so.26...done.
Loaded symbols for /usr/lib/libgnutls.so.26
Reading symbols from /usr/lib/libtasn1.so.3...done.
Loaded symbols for /usr/lib/libtasn1.so.3
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /usr/lib/libgcrypt.so.11...done.
Loaded symbols for /usr/lib/libgcrypt.so.11
Reading symbols from /usr/lib/libgpg-error.so.0...done.
Loaded symbols for /usr/lib/libgpg-error.so.0
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libltdl.so.3...done.
Loaded symbols for /usr/lib/libltdl.so.3
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libwrap.so.0...done.
Loaded symbols for /lib/libwrap.so.0
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/libstdc++.so.6...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /usr/lib/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib/sasl2/libanonymous.so.2
Reading symbols from /usr/lib/sasl2/liblogin.so.2...done.
Loaded symbols for /usr/lib/sasl2/liblogin.so.2
Reading symbols from /usr/lib/sasl2/libntlm.so.2...done.
Loaded symbols for /usr/lib/sasl2/libntlm.so.2
Reading symbols from /usr/lib/i686/cmov/libcrypto.so.0.9.8...done.
Loaded symbols for /usr/lib/i686/cmov/libcrypto.so.0.9.8
Reading symbols from /usr/lib/sasl2/libdigestmd5.so.2...done.
Loaded symbols for /usr/lib/sasl2/libdigestmd5.so.2
Reading symbols from /usr/lib/sasl2/libsasldb.so.2...done.
Loaded symbols for /usr/lib/sasl2/libsasldb.so.2
Reading symbols from /usr/lib/sasl2/libcrammd5.so.2...done.
Loaded symbols for /usr/lib/sasl2/libcrammd5.so.2
Reading symbols from /usr/lib/sasl2/libgssapiv2.so.2...done.
Loaded symbols for /usr/lib/sasl2/libgssapiv2.so.2
Reading symbols from /usr/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /usr/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libcom_err.so.2...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libkrb5support.so.0...done.
Loaded symbols for /usr/lib/libkrb5support.so.0
Reading symbols from /lib/libkeyutils.so.1...done.
Loaded symbols for /lib/libkeyutils.so.1
Reading symbols from /usr/lib/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib/sasl2/libplain.so.2
Reading symbols from /sysvol/ldap/release/2.4.19/lib/ldap/back_bdb-2.4.so.2...done.
Loaded symbols for /sysvol/ldap/release/2.4.19/lib/ldap/back_bdb-2.4.so.2
Reading symbols from /sysvol/ldap/release/2.4.19/lib/ldap/syncprov-2.4.so.2...done.
Loaded symbols for /sysvol/ldap/release/2.4.19/lib/ldap/syncprov-2.4.so.2
Reading symbols from /sysvol/ldap/release/2.4.19/lib/ldap/ppolicy-2.4.so.2...done.
Loaded symbols for /sysvol/ldap/release/2.4.19/lib/ldap/ppolicy-2.4.so.2
0xb7df3492 in pthread_join () from /lib/libpthread.so.0
(gdb) continue
Continuing.
[New Thread 0xa0c42b90 (LWP 15033)]

Program received signal SIGINT, Interrupt.
[Switching to Thread 0xb6dc79b0 (LWP 14884)]
0xb7df3492 in pthread_join () from /lib/libpthread.so.0
(gdb) backtracer  full
#0  0xb7df3492 in pthread_join () from /lib/libpthread.so.0
No symbol table info available.
#1  0xb7fb9264 in ldap_pvt_thread_join (thread=2706652048, thread_return=0x0) at thr_posix.c:197
No locals.
#2  0x080715e3 in slapd_daemon ()
No symbol table info available.
#3  0x0805e535 in main ()
No symbol table info available.
(gdb) info registers
eax            0xfffffe00	-512
ecx            0x0	0
edx            0x3a25	14885
ebx            0xa1543bd8	-1588315176
esp            0xbfff9274	0xbfff9274
ebp            0xbfff92a8	0xbfff92a8
esi            0x0	0
edi            0xb7e01ff4	-1210048524
eip            0xb7df3492	0xb7df3492 <pthread_join+274>
eflags         0x200246	[ PF ZF IF ID ]
cs             0x73	115
ss             0x7b	123
ds             0x7b	123
es             0x7b	123
fs             0x0	0
gs             0x33	51
(gdb) thread apply all backtrace

Thread 4 (Thread 0xa0c42b90 (LWP 15033)):
#0  0xb7df68f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#1  0xb7fb9124 in ldap_pvt_thread_cond_wait (cond=0x9ca3324, mutex=0x9ca330c) at thr_posix.c:277
#2  0xb7fb84d2 in ldap_int_thread_pool_wrapper (xpool=0x9ca3308) at tpool.c:672
#3  0xb7df2f3b in start_thread () from /lib/libpthread.so.0
#4  0xb6fb4bee in clone () from /lib/libc.so.6

Thread 3 (Thread 0xa1543b90 (LWP 14885)):
#0  0xb6fb539c in epoll_wait () from /lib/libc.so.6
#1  0x080748ff in ?? ()
#2  0x00000006 in ?? ()
#3  0x09c846c0 in ?? ()
#4  0x00000400 in ?? ()
#5  0x000bb800 in ?? ()
#6  0x00000000 in ?? ()

Thread 2 (Thread 0xa1143b90 (LWP 14886)):
#0  0xb7df68f0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#1  0xb7fb9124 in ldap_pvt_thread_cond_wait (cond=0x9ca3324, mutex=0x9ca330c) at thr_posix.c:277
#2  0xb7fb84d2 in ldap_int_thread_pool_wrapper (xpool=0x9ca3308) at tpool.c:672
#3  0xb7df2f3b in start_thread () from /lib/libpthread.so.0
#4  0xb6fb4bee in clone () from /lib/libc.so.6

Thread 1 (Thread 0xb6dc79b0 (LWP 14884)):
#0  0xb7df3492 in pthread_join () from /lib/libpthread.so.0
#1  0xb7fb9264 in ldap_pvt_thread_join (thread=2706652048, thread_return=0x0) at thr_posix.c:197
#2  0x080715e3 in slapd_daemon ()
#3  0x0805e535 in main ()
(gdb) quit
The program is running.  Quit anyway (and detach it)? (y or n) Detaching from program: /sysvol/ldap/release/2.4.19/lib/slapd, process 14884

--------------030103000807090505050604--
Prev by Date: Re: (ITS#6334) hang during ldapmodify
Next by Date: Re: (ITS#6334) hang during ldapmodify
Index(es):
- Chronological
- Thread