[Date Prev][Date Next] [Chronological] [Thread] [Top]
(ITS#7740) back-meta lock threads when remote node is unreachable

To: openldap-its@OpenLDAP.org
Subject: (ITS#7740) back-meta lock threads when remote node is unreachable
From: jorge.perez.burgos@ericsson.com
Date: Tue, 5 Nov 2013 09:21:33 GMT
Auto-submitted: auto-generated (OpenLDAP-ITS)
Full_Name: Jorge Perez Burgos
Version: 2.4.35
OS: SLES 11
URL: ftp://ftp.openldap.org/incoming/0001-Don-t-lock-other-threads-while-trying-to-connect-to.patch
Submission from: (NULL) (195.235.15.243)


When there is meta proxy traffic to another node and this node becomes
unreachable the thread in meta retrying to connect to the remote node locks all
the other threads making the slapd unresponsive until the timeout expires.
Attach a possible fix
ftp://ftp.openldap.org/incoming/0001-Don-t-lock-other-threads-while-trying-to-connect-to.patch

and a backtraceo of some threads when the problem happens:

Thread 259 (Thread 0x7ff9a771e700 (LWP 27085)):
#0  0x00007ff9fc2a6294 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007ff9fc2a1619 in _L_lock_1008 () from /lib64/libpthread.so.0
#2  0x00007ff9fc2a142e in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004b6007 in meta_back_search (op=0x31a8800, rs=0x7ff9a771d930) at
search.c:2012
#4  0x000000000043ab31 in fe_op_search (op=0x31a8800, rs=0x7ff9a771d930) at
search.c:402
#5  0x00000000004a0ef7 in overlay_op_walk (op=0x31a8800, rs=0x7ff9a771d930,
which=op_search, oi=0x87cc000, on=0x0) at backover.c:671
#6  0x00000000004a1970 in over_op_func (op=0x31a8800, rs=0x7ff9a771bdc8,
which=op_bind) at backover.c:723
#7  0x000000000043b37c in do_search (op=0x31a8800, rs=0x7ff9a771d930) at
search.c:247
#8  0x0000000000437eef in connection_operation (ctx=0x7ff9a771db30,
arg_v=<optimized out>) at connection.c:1313
#9  0x0000000000438b1b in connection_read_thread (ctx=0x7ff9a771db30,
argv=<optimized out>) at connection.c:1449
#10 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>)
at tpool.c:688
#11 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0
#12 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6
#13 0x0000000000000000 in ?? ()

226 Transfer complete (unique file
name:0001-Don-t-lock-other-threads-while-trying-to-connect-to.patch).

Thread 258 (Thread 0x7ff9a6f1d700 (LWP 27086)):
#0  0x00007ff9fbff1786 in poll () from /lib64/libc.so.6
#1  0x0000000000504b19 in ldap_int_poll (ld=0x5f03e030, s=462,
tvp=0x7ff9a6f1afd0, wr=<optimized out>) at os-ip.c:302
#2  0x0000000000505248 in ldap_pvt_connect (async=<optimized out>,
addrlen=<optimized out>, sin=<optimized out>, s=<optimized out>, ld=<optimized
out>) at os-ip.c:465
#3  ldap_connect_to_host (ld=0x5f03e030, sb=0x592c0ed0, proto=1, srv=0x6e26edc0,
async=0) at os-ip.c:658
#4  0x00000000004ef4d7 in ldap_int_open_connection (ld=0x5f03e030,
conn=0x574bfc20, srv=0x6e26edc0, async=0) at open.c:367
#5  0x000000000050243a in ldap_new_connection (ld=0x5f03e030,
srvlist=0x6cf79598, use_ldsb=1, connect=<optimized out>, bind=0x0, m_req=0,
m_res=0) at request.c:481
#6  0x00000000004ef41f in ldap_open_defconn (ld=0x7ff9a6f1aef0) at open.c:41
#7  0x0000000000503158 in ldap_send_initial_request (ld=0x5f03e030,
msgtype=<optimized out>, dn=<optimized out>, ber=0x6db20780, msgid=1) at
request.c:130
#8  0x00000000004f8ebc in ldap_sasl_bind (ld=0x5f03e030, dn=0x89e9660 <error
reading variable>, mechanism=0x0, cred=0x7ff9a6f1b220, sctrls=0x0,
cctrls=<optimized out>, msgidp=0x7ff9a6f1b248) at sasl.c:148
#9  0x00000000004cb270 in meta_back_proxy_authz_bind (mc=0x12165ba0,
candidate=2, op=0x342ac00, rs=0x7ff9a6f1ca60, sendok=LDAP_BACK_SENDERR,
dolock=0) at bind.c:1582
#10 0x00000000004cb49f in meta_back_single_dobind (op=0x342ac00,
rs=0x7ff9a6f1ca60, mcp=0x7ff9a6f1b560, candidate=2, sendok=LDAP_BACK_SENDERR,
nretries=<optimized out>, dolock=0) at bind.c:605
#11 0x00000000004d5df3 in meta_back_retry (op=0x342ac00, rs=0x7ff9a6f1ca60,
mcp=0x7ff9a6f1b560, candidate=2, sendok=LDAP_BACK_SENDERR) at conn.c:766
#12 0x00000000004b42c5 in meta_back_modify (op=0x342ac00, rs=0x7ff9a6f1ca60) at
modify.c:194
#13 0x00000000004506d8 in fe_op_modify (op=0x342ac00, rs=0x7ff9a6f1ca60) at
modify.c:303
#14 0x00000000004a0ef7 in overlay_op_walk (op=0x342ac00, rs=0x7ff9a6f1ca60,
which=op_modify, oi=0x87cc000, on=0x0) at backover.c:671
#15 0x00000000004a1970 in over_op_func (op=0x342ac00, rs=0x7ff9a6f1a918,
which=4000) at backover.c:723
#16 0x0000000000451007 in do_modify (op=0x342ac00, rs=0x7ff9a6f1ca60) at
modify.c:177
#17 0x0000000000437eef in connection_operation (ctx=0x7ff9a6f1cb30,
arg_v=<optimized out>) at connection.c:1313
#18 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>)
at tpool.c:688
#19 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0
#20 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6
#21 0x0000000000000000 in ?? ()

Thread 257 (Thread 0x7ff9a671c700 (LWP 27088)):
#0  0x00007ff9fc2a6294 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007ff9fc2a1619 in _L_lock_1008 () from /lib64/libpthread.so.0
#2  0x00007ff9fc2a142e in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004d4540 in meta_back_getconn (op=0x46fe000, rs=0x7ff9a671b930,
candidate=0x0, sendok=LDAP_BACK_SENDERR) at conn.c:1129
#4  0x00000000004b5b61 in meta_back_search (op=0x46fe000, rs=0x7ff9a671b930) at
search.c:803
#5  0x000000000043ab31 in fe_op_search (op=0x46fe000, rs=0x7ff9a671b930) at
search.c:402
#6  0x00000000004a0ef7 in overlay_op_walk (op=0x46fe000, rs=0x7ff9a671b930,
which=op_search, oi=0x87cc000, on=0x0) at backover.c:671
#7  0x00000000004a1970 in over_op_func (op=0x46fe000, rs=0x0, which=op_bind) at
backover.c:723
#8  0x000000000043b37c in do_search (op=0x46fe000, rs=0x7ff9a671b930) at
search.c:247
#9  0x0000000000437eef in connection_operation (ctx=0x7ff9a671bb30,
arg_v=<optimized out>) at connection.c:1313
#10 0x0000000000438b1b in connection_read_thread (ctx=0x7ff9a671bb30,
argv=<optimized out>) at connection.c:1449
#11 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>)
at tpool.c:688
#12 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0
#13 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6
#14 0x0000000000000000 in ?? ()

Thread 256 (Thread 0x7ff9a5f1b700 (LWP 27089)):
#0  0x00007ff9fc2a6294 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007ff9fc2a1619 in _L_lock_1008 () from /lib64/libpthread.so.0
#2  0x00007ff9fc2a142e in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004d4540 in meta_back_getconn (op=0x46f5000, rs=0x7ff9a5f1a930,
candidate=0x0, sendok=LDAP_BACK_SENDERR) at conn.c:1129
#4  0x00000000004b5b61 in meta_back_search (op=0x46f5000, rs=0x7ff9a5f1a930) at
search.c:803
#5  0x000000000043ab31 in fe_op_search (op=0x46f5000, rs=0x7ff9a5f1a930) at
search.c:402
#6  0x00000000004a0ef7 in overlay_op_walk (op=0x46f5000, rs=0x7ff9a5f1a930,
which=op_search, oi=0x87cc000, on=0x0) at backover.c:671
#7  0x00000000004a1970 in over_op_func (op=0x46f5000, rs=0x0, which=op_bind) at
backover.c:723
#8  0x000000000043b37c in do_search (op=0x46f5000, rs=0x7ff9a5f1a930) at
search.c:247
#9  0x0000000000437eef in connection_operation (ctx=0x7ff9a5f1ab30,
arg_v=<optimized out>) at connection.c:1313
#10 0x0000000000438b1b in connection_read_thread (ctx=0x7ff9a5f1ab30,
argv=<optimized out>) at connection.c:1449
#11 0x00000000004ee750 in ldap_int_thread_pool_wrapper (xpool=<optimized out>)
at tpool.c:688
#12 0x00007ff9fc29f7b6 in start_thread () from /lib64/libpthread.so.0
#13 0x00007ff9fbffac5d in clone () from /lib64/libc.so.6
#14 0x0000000000000000 in ?? ()
Prev by Date: Re: (ITS#7735) Memory leaks on Multi-Master replications
Next by Date: Re: (ITS#7740) back-meta lock threads when remote node is unreachable
Index(es):
- Chronological
- Thread