[Date Prev][Date Next] [Chronological] [Thread] [Top]
(ITS#4640) slapd lockup

To: openldap-its@OpenLDAP.org
Subject: (ITS#4640) slapd lockup
From: quanah@stanford.edu
Date: Wed, 16 Aug 2006 08:33:14 GMT
Full_Name: Quanah Gibson-Mount
Version: 2.3.25
OS: Linux 2.6 kernel
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (171.66.155.86)


slapd locked up, backtrace shows:

(gdb) info threads
  10 Thread 1082128752 (LWP 19637)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  9 Thread 1090517360 (LWP 19638)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  8 Thread 1098905968 (LWP 19639)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  7 Thread 1107294576 (LWP 19640)  ldap_int_thread_pool_wrapper (xpool=0x636178)
at tpool.c:436
  6 Thread 1115683184 (LWP 19769)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  5 Thread 1124071792 (LWP 19770)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  4 Thread 1132460400 (LWP 19771)  __memp_sync_int (dbenv=0x7c19f0, dbmfp=0x0,
trickle_max=0, op=DB_SYNC_CACHE, wrotep=0x0) at ../dist/../mp/mp_sync.c:251
  3 Thread 1140849008 (LWP 19772)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  2 Thread 1149237616 (LWP 20127)  0x00002af45ce637c4 in __lll_mutex_lock_wait
() from /lib/libpthread.so.0
  1 Thread 47229024698160 (LWP 19635)  0x00002af45ce5fe2c in pthread_join ()
from /lib/libpthread.so.0



[Switching to thread 4 (Thread 1132460400 (LWP 19771))]#0  __memp_sync_int
(dbenv=0x7c19f0, dbmfp=0x0, trickle_max=0, op=DB_SYNC_CACHE, wrotep=0x0)
    at ../dist/../mp/mp_sync.c:251
251     ../dist/../mp/mp_sync.c: No such file or directory.
        in ../dist/../mp/mp_sync.c
(gdb) bt
#0  __memp_sync_int (dbenv=0x7c19f0, dbmfp=0x0, trickle_max=0, op=DB_SYNC_CACHE,
wrotep=0x0) at ../dist/../mp/mp_sync.c:251
#1  0x00002af45dd792eb in __memp_sync (dbenv=0x7c19f0, lsnp=0x0) at
../dist/../mp/mp_sync.c:99
#2  0x00002af45dd7e232 in __txn_checkpoint (dbenv=0x7c19f0, kbytes=1024,
minutes=5, flags=60) at ../dist/../txn/txn.c:1387
#3  0x00002af45dd7e398 in __txn_checkpoint_pp (dbenv=0x7c19f0, kbytes=1024,
minutes=5, flags=0) at ../dist/../txn/txn.c:1286
#4  0x00002af45e0b962c in hdb_modify (op=0x437fee80, rs=0x437fec00) at
modify.c:594
#5  0x0000000000472959 in do_syncrep2 (op=0x437fee80, si=0x71e300) at
syncrepl.c:1374
#6  0x0000000000472ed3 in do_syncrepl (ctx=0x437ff4e0, arg=0x2af55bcd1a80) at
syncrepl.c:1085
#7  0x00000000004286a2 in connection_read_thread (ctx=0x437ff4e0, argv=0x0) at
connection.c:1218
#8  0x00002af45c319522 in ldap_int_thread_pool_wrapper (xpool=0x2af55bcd1a80) at
tpool.c:478
#9  0x00002af45ce5eb55 in start_thread () from /lib/libpthread.so.0
#10 0x00002af45d0407f0 in clone () from /lib/libc.so.6
#11 0x0000000000000000 in ?? ()
#12 0x0000000000000000 in ?? ()
#13 0x0000000000000000 in ?? ()
#14 0x0000000000000000 in ?? ()
#15 0x0000000000000000 in ?? ()
#16 0x0000000000000000 in ?? ()
#17 0x0000000000000000 in ?? ()
#18 0x0000000000000000 in ?? ()
#19 0x00002af45d18fe00 in __check_rhosts_file () from /lib/libc.so.6
#20 0x00000000437ffdc8 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x00002af45d0799a0 in ?? () from /lib/libc.so.6
#23 0x00002af45d07a1a0 in ?? () from /lib/libc.so.6
#24 0x00002af45d07a7a0 in ?? () from /lib/libc.so.6
#25 0x00002aab1e900020 in ?? ()
#26 0x0000000000000001 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x00000000437ff970 in ?? ()
#30 0x00000000098bda48 in ?? ()
#31 0x00000000437ff970 in ?? ()
#32 0x0000000000000001 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000000000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0x0000000000000000 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000000000000 in ?? ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000000 in ?? ()
#44 0x0000000000000000 in ?? ()
#45 0x0000000042fff9f0 in ?? ()
#46 0x0000000043fff9f0 in ?? ()
#47 0x0000000000004d3b in ?? ()
#48 0x0000000000000000 in ?? ()
#49 0x00000000437ff830 in ?? ()
#50 0x0000000000000000 in ?? ()
#51 0x0000000000000000 in ?? ()
#52 0x0000000000000000 in ?? ()
#53 0x0000000000000000 in ?? ()
#54 0x0000000000000000 in ?? ()
#55 0x0000000000000001 in ?? ()



[Switching to thread 7 (Thread 1107294576 (LWP 19640))]#0 
ldap_int_thread_pool_wrapper (xpool=0x636178) at tpool.c:436
436     tpool.c: No such file or directory.
        in tpool.c
(gdb) bt
#0  ldap_int_thread_pool_wrapper (xpool=0x636178) at tpool.c:436
#1  0x00002af45ce5eb55 in start_thread () from /lib/libpthread.so.0
#2  0x00002af45d0407f0 in clone () from /lib/libc.so.6
#3  0x0000000000000000 in ?? ()
#4  0x0000000000000000 in ?? ()
#5  0x0000000000000000 in ?? ()
#6  0x0000000000000000 in ?? ()
#7  0x0000000000000000 in ?? ()
#8  0x0000000000000000 in ?? ()
#9  0x0000000000000000 in ?? ()
#10 0x0000000000000000 in ?? ()
#11 0x00002af45d18fe00 in __check_rhosts_file () from /lib/libc.so.6
#12 0x0000000041fffdc8 in ?? ()
#13 0x000000000000000b in ?? ()
#14 0x00002af45d0799a0 in ?? () from /lib/libc.so.6
#15 0x00002af45d07a1a0 in ?? () from /lib/libc.so.6
#16 0x00002af45d07a7a0 in ?? () from /lib/libc.so.6
#17 0x00002aaaabe00020 in ?? ()
#18 0x0000000000000001 in ?? ()
#19 0x0000000000000000 in ?? ()
#20 0x0000000000000000 in ?? ()
#21 0x0000000041fff970 in ?? ()
#22 0x00002aaaabe02638 in ?? ()
#23 0x0000000041fff970 in ?? ()
#24 0x0000000000000001 in ?? ()
#25 0x0000000000000000 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x00000000417ff9f0 in ?? ()
#38 0x00000000427ff9f0 in ?? ()
#39 0x0000000000004cb8 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000041fff830 in ?? ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000000 in ?? ()
#44 0x0000000000000000 in ?? ()
#45 0x0000000000000000 in ?? ()
#46 0x0000000000000000 in ?? ()
#47 0x0000000000000001 in ?? ()
#48 0x00000000007f3d60 in ?? ()
#49 0x0000000000000000 in ?? ()
#50 0x0000000000000000 in ?? ()
#51 0x0000000000000000 in ?? ()
#52 0x0000000000000000 in ?? ()
#53 0x0000000000000000 in ?? ()
#54 0x0000000000000000 in ?? ()
#55 0x0000000000000000 in ?? ()



[Switching to thread 9 (Thread 1090517360 (LWP 19638))]#0  0x00002af45ce637c4 in
__lll_mutex_lock_wait () from /lib/libpthread.so.0
(gdb) bt
#0  0x00002af45ce637c4 in __lll_mutex_lock_wait () from /lib/libpthread.so.0
#1  0x0000000000636178 in ?? ()
#2  0x0000000000000008 in ?? ()
#3  0x00002af45ce60971 in pthread_mutex_lock () from /lib/libpthread.so.0
#4  0x0000000000000000 in ?? ()
#5  0x0000000040fff4e0 in ?? ()
#6  0x000000bc00000008 in ?? ()
#7  0x000000000000003d in ?? ()
#8  0x0000000000000000 in ?? ()
#9  0x0000000000000000 in ?? ()
#10 0x0000000000000000 in ?? ()
#11 0x0000000000000000 in ?? ()
#12 0x0000000000000000 in ?? ()
#13 0x0000000000000000 in ?? ()
#14 0x0000000000636178 in ?? ()
#15 0x0000000033561ae0 in ?? ()
#16 0x00000000000002a8 in ?? ()
#17 0x0000000000636170 in ?? ()
#18 0x0000000000636178 in ?? ()
#19 0x00002af45c448740 in tid_zero () from /usr/local/lib/libldap_r-2.3.so.0
#20 0x0000000040fff4e0 in ?? ()
#21 0x00002af45c31952a in ldap_int_thread_pool_wrapper (xpool=0x636178) at
tpool.c:480
#22 0x00002af45ce5eb55 in start_thread () from /lib/libpthread.so.0
#23 0x00002af45d0407f0 in clone () from /lib/libc.so.6
#24 0x0000000000000000 in ?? ()
#25 0x0000000000000000 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x00002af45d18fe00 in __check_rhosts_file () from /lib/libc.so.6
#33 0x0000000040fffdc8 in ?? ()
#34 0x000000000000000b in ?? ()
#35 0x00002af45d0799a0 in ?? () from /lib/libc.so.6
#36 0x00002af45d07a1a0 in ?? () from /lib/libc.so.6
#37 0x00002af45d07a7a0 in ?? () from /lib/libc.so.6
#38 0x00002af45d1a9580 in __after_morecore_hook () from /lib/libc.so.6
#39 0x0000000000000001 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000000000000 in ?? ()
#42 0x0000000040fff970 in ?? ()
#43 0x000000000079d078 in ?? ()
#44 0x0000000040fff970 in ?? ()
#45 0x0000000000000001 in ?? ()
#46 0x0000000000000000 in ?? ()
#47 0x0000000000000000 in ?? ()
#48 0x0000000000000000 in ?? ()
#49 0x0000000000000000 in ?? ()
#50 0x0000000000000000 in ?? ()
#51 0x0000000000000000 in ?? ()
#52 0x0000000000000000 in ?? ()
#53 0x0000000000000000 in ?? ()
#54 0x0000000000000000 in ?? ()
#55 0x0000000000000000 in ?? ()
---Type <return> to continue, or q <return> to quit---q


The slapd 256 level log shows:

Aug 16 01:02:56 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:02:56 ldap4 last message repeated 5260 times
Aug 16 01:02:56 ldap4 slapd[19635]: conn=71425 op=73 SEARCH RESULT tag=101 err=0
nentries=1 text=
Aug 16 01:02:56 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:03:27 ldap4 last message repeated 496443 times
Aug 16 01:03:48 ldap4 last message repeated 409334 times
Aug 16 01:03:48 ldap4 slapd[19635]: conn=125409 op=91 SRCH
base="cn=people,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=samsingh)"
Aug 16 01:03:49 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:03:49 ldap4 last message repeated 9578 times
Aug 16 01:03:49 ldap4 slapd[19635]: conn=125409 op=91 SEARCH RESULT tag=101
err=0 nentries=1 text=
Aug 16 01:03:49 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:04:20 ldap4 last message repeated 531175 times
Aug 16 01:05:21 ldap4 last message repeated 778102 times
Aug 16 01:06:20 ldap4 last message repeated 1431826 times
Aug 16 01:06:20 ldap4 slapd[19635]: conn=107699 op=75 SRCH
base="cn=people,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=jaiswala)"
Aug 16 01:06:20 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:06:21 ldap4 last message repeated 11629 times
Aug 16 01:06:21 ldap4 slapd[19635]: conn=107699 op=75 SEARCH RESULT tag=101
err=0 nentries=1 text=
Aug 16 01:06:21 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:06:52 ldap4 last message repeated 855685 times
Aug 16 01:07:53 ldap4 last message repeated 1515673 times
Aug 16 01:08:54 ldap4 last message repeated 1422976 times
Aug 16 01:09:55 ldap4 last message repeated 1202434 times
Aug 16 01:10:56 ldap4 last message repeated 1702411 times
Aug 16 01:11:57 ldap4 last message repeated 1499870 times
Aug 16 01:12:58 ldap4 last message repeated 1565934 times
Aug 16 01:13:59 ldap4 last message repeated 1431042 times
Aug 16 01:15:00 ldap4 last message repeated 1111519 times
Aug 16 01:16:01 ldap4 last message repeated 1249596 times
Aug 16 01:17:02 ldap4 last message repeated 1411805 times
Aug 16 01:18:03 ldap4 last message repeated 1411947 times
Aug 16 01:19:04 ldap4 last message repeated 1394456 times
Aug 16 01:20:05 ldap4 last message repeated 1371976 times
Aug 16 01:21:06 ldap4 last message repeated 1240913 times
Aug 16 01:22:03 ldap4 last message repeated 1278752 times
Aug 16 01:22:03 ldap4 slapd[19635]: conn=129354 op=7 SRCH
base="cn=people,dc=stanford,dc=edu" scope=2 deref=0 filter="(uid=plabo)"
Aug 16 01:22:03 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:22:04 ldap4 last message repeated 34046 times
Aug 16 01:22:04 ldap4 slapd[19635]: conn=129354 op=7 SEARCH RESULT tag=101 err=0
nentries=1 text=
Aug 16 01:22:04 ldap4 slapd[19635]: connection_read(22): no connection!
Aug 16 01:22:35 ldap4 last message repeated 850797 times
Aug 16 01:23:36 ldap4 last message repeated 958223 times
Aug 16 01:24:37 ldap4 last message repeated 701951 times
Aug 16 01:25:38 ldap4 last message repeated 858713 times
Aug 16 01:26:39 ldap4 last message repeated 740839 times
Aug 16 01:27:40 ldap4 last message repeated 672820 times
Aug 16 01:28:40 ldap4 last message repeated 711794 times
Aug 16 01:29:41 ldap4 last message repeated 503893 times
Aug 16 01:29:58 ldap4 last message repeated 78487 times
Aug 16 01:32:09 ldap4 slapd[19635]: connection_read(22): no connection!
Prev by Date: Re: (ITS#4639) slurpd parses attribute named delete incorrectly
Next by Date: (ITS#4314) C API for paged results
Index(es):
- Chronological
- Thread