Re: (ITS#6089) connection_destroy() c->c_writers == 0 assertion failure

[hyc@symas.com]

jwm@horde.net wrote:
> Poked around a bit in the core:
>
> (gdb) print c->c_writers
> $1 = -1
> (gdb) print c->c_pending_ops->stqh_last[0]
> $5 = (struct Operation *) 0x0
> (gdb) print c->c_n_ops_pending
> $6 = 0
>
> So there are no pending ops on this connection, but c_writers == -1
> indicates that one blocked writer remains to notice that the connection has
> been closed.
>
> I looked at send_ldap_ber() and can't immediately find fault with its
> manipulation of c_writers. connection_closing() wakes up all blocked
> writers, so by the time connection_close() calls connection_destroy() and
> this assertion is checked, c_writers should be 0. Furthermore,
> c->c_conn_state is SLAP_C_CLOSING (0x4), which can only happen in
> connection_closing().
>
> This is about as far as I can get; any other ideas?

I'm stumped. The only way out of connection_closing() is for c_writers to go 
to zero. And from your core file, there are no other threads blocked on any of 
the write mutexes or condition variables. You can reproduce this easily?

> john
>
>
> (gdb) print *c
> $2 = {c_struct_state = 2, c_conn_state = 4, c_conn_idx = 1878, c_sd = 1878,
>    c_close_reason = 0x0, c_mutex = {__data = {__lock = 1, __count = 0,
>        __owner = 9540, __kind = 0, __nusers = 1, {__spins = 0, __list = {
>            __next = 0x0}}},
>      __size = "\001\000\000\000\000\000\000\000D%\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 1}, c_sb = 0x175c7240,
>    c_starttime = 1240942246, c_activitytime = 1240942264, c_connid = 73097614,
>    c_peer_domain = {bv_len = 7, bv_val = 0x137e02c8 "unknown"}, c_peer_name = {
>      bv_len = 23, bv_val = 0x191c0be0 "IP=172.16.182.103:50023"},
>    c_listener = 0x9bb7370, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0},
>    c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0,
>      bv_val = 0x0}, c_authz_backend = 0x9bba500, c_authz_cookie = 0x0,
>    c_authz = {sai_method = 128, sai_mech = {bv_len = 0, bv_val = 0x0},
>      sai_dn = {bv_len = 27, bv_val = 0x1876bd20 "cn=nsspluspasswd,o=example2"},
>      sai_ndn = {bv_len = 27,
>        bv_val = 0x189f5780 "cn=nsspluspasswd,o=example2"}, sai_ssf = 128,
>      sai_transport_ssf = 0, sai_tls_ssf = 128, sai_sasl_ssf = 0},
>    c_protocol = 3, c_ops = {stqh_first = 0x0, stqh_last = 0x9d7df18},
>    c_pending_ops = {stqh_first = 0x0, stqh_last = 0x9d7df20}, c_write1_mutex = {
>      __data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, {
>          __spins = 0, __list = {__next = 0x0}}},
>      __size = '\0'<repeats 23 times>, __align = 0}, c_write1_cv = {__data = {
>        __lock = 0, __futex = 2, __total_seq = 1, __wakeup_seq = 1,
>        __woken_seq = 1, __mutex = 0x9d7df28, __nwaiters = 0,
>        __broadcast_seq = 1},
>      __size = "\000\000\000\000\002\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000(##\t\000\000\000\000\001\000\000\000\000\000\000", __align = 8589934592}, c_write2_mutex = {
>      __data = {__lock = 0, __count = 0, __owner = 0, __kind = 0, __nusers = 0, {
>          __spins = 0, __list = {__next = 0x0}}},
>      __size = '\0'<repeats 23 times>, __align = 0}, c_write2_cv = {__data = {
>        __lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0,
>        __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0},
>      __size = '\0'<repeats 47 times>, __align = 0}, c_currentber = 0x0,
>    c_writers = -1, c_sasl_bind_in_progress = 0 '\0', c_writewaiter = 0 '\0',
>    c_is_tls = 1 '\001', c_needs_tls_accept = 0 '\0', c_sasl_layers = 0 '\0',
>    c_sasl_done = 0 '\0', c_sasl_authctx = 0x13497200, c_sasl_sockctx = 0x0,
>    c_sasl_extra = 0x14d956c0, c_sasl_bindop = 0x0, c_pagedresults_state = {
>      ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {
>        bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 6, c_n_ops_executing = 0,
>    c_n_ops_pending = 0, c_n_ops_completed = 6, c_n_get = 4, c_n_read = 4,
>    c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0, c_clientarg = 0x0,
>    c_send_ldap_result = 0x808aea0<slap_send_ldap_result>,
>    c_send_search_entry = 0x80885e0<slap_send_search_entry>,
>    c_send_search_reference = 0x8087da0<slap_send_search_reference>,
>    c_send_ldap_extended = 0x808a910<slap_send_ldap_extended>,
>    c_send_ldap_intermediate = 0x808a760<slap_send_ldap_intermediate>}
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/