qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Ross <tr...@redhat.com>
Subject Re: proton segfault on freeing collector? (was Re: qpidd 0.32 crashes)
Date Thu, 02 Jul 2015 14:05:01 GMT


On 07/01/2015 03:27 PM, Gordon Sim wrote:
> On 06/30/2015 10:17 AM, Michael Ivanov wrote:
>> Ok it crashed again and here are the backtrace and protocol trace:
>
> The error occurs when draining events on the collector as part of
> freeing it. The connection has already been 'freed' by the application,
> but is referenced by an event in the collector. My understanding is the
> reference counting scheme used by proton is supposed to handle this.
>
> Unfortunately I'm still unable to reproduce this. I have tried both
> simple tests that generate events then free the connection then the
> associated collector with events still in it and that all seems to work
> fine. I've also tried some stress tests where the transport is killed
> while the client is doing various things (that seems to be what the
> protocol trace indicates, since there is no explicit close).
>
> The broker could free the collector *before* freeing the connection if
> that was required, but from what I can tell it should make no difference.

For what it's worth, there was a crash in Dispatch which seemed to be 
caused by freeing the collector before the connection.  See 
https://svn.apache.org/viewvc?view=revision&revision=1588363

>
> Can anyone more familiar with proton internals spot anything odd with
> regard to the connection state below (after stack trace)? The apparent
> location of the segfault is when calling pn_refcount on the connection.
> The only reason I can see for that is if the 'head' containing class and
> reference count was no longer valid.
>
> Is it possible to reproduce when running the broker under valgrind?
> (Sorry to keep asking for more things, but without a reproducer and
> extensive knowledge of proton, its hard to figure out what the issue
> might be).
>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x000000000115bc00 in ?? ()
>> (gdb) bt
>> #0  0x000000000115bc00 in ?? ()
>> #1  0x00007fb7c6afc74d in pn_connection_finalize (object=0xda2c90) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/engine/engine.c:467
>> #2  0x00007fb7c6af15c8 in pn_class_decref (clazz=0x7fb7c6d25400
>> <clazz.4691>, object=0xda2c90)
>>      at /usr/src/debug/qpid-proton-0.9/proton-c/src/object/object.c:97
>> #3  0x00007fb7c6af15c8 in pn_class_decref (clazz=0x7fb7c6d25380
>> <clazz.4874>, object=0xebfc90)
>>      at /usr/src/debug/qpid-proton-0.9/proton-c/src/object/object.c:97
>> #4  0x00007fb7c6af15c8 in pn_class_decref (clazz=0x7fb7c6d25300
>> <clazz.4927>, object=0x1156450)
>>      at /usr/src/debug/qpid-proton-0.9/proton-c/src/object/object.c:97
>> #5  0x00007fb7c6aff310 in pn_event_finalize (event=0xe33710) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/events/event.c:190
>> #6  pn_event_finalize_cast (object=0xe33710) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/events/event.c:235
>> #7  0x00007fb7c6af15c8 in pn_class_decref (clazz=0x7fb7c6d25480
>> <clazz.2269>, object=0xe33710)
>>      at /usr/src/debug/qpid-proton-0.9/proton-c/src/object/object.c:97
>> #8  0x00007fb7c6af17f2 in pn_decref (object=<optimized out>) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/object/object.c:252
>> #9  0x00007fb7c6aff4b2 in pn_collector_pop
>> (collector=collector@entry=0xdac360) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/events/event.c:167
>> #10 0x00007fb7c6aff508 in pn_collector_drain (collector=0xdac360) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/events/event.c:34
>> #11 pn_collector_release (collector=collector@entry=0xdac360) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/events/event.c:96
>> #12 0x00007fb7c6aff529 in pn_collector_free (collector=0xdac360) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/events/event.c:87
>> #13 0x00007fb7c6d79669 in qpid::broker::amqp::Connection::~Connection
>> (this=0xda2ec0, __in_chrg=<optimized out>)
>>      at
>> /usr/src/debug/qpid-cpp-0.32/src/qpid/broker/amqp/Connection.cpp:171
>> #14 0x00007fb7c6d79819 in qpid::broker::amqp::Connection::~Connection
>> (this=0xda2ec0, __in_chrg=<optimized out>)
>>      at
>> /usr/src/debug/qpid-cpp-0.32/src/qpid/broker/amqp/Connection.cpp:173
>> #15 0x00007fb7ca57fd63 in qpid::sys::AsynchIOHandler::~AsynchIOHandler
>> (this=0xda2920, __in_chrg=<optimized out>)
>>      at /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/AsynchIOHandler.cpp:73
>> #16 0x00007fb7ca57fe59 in qpid::sys::AsynchIOHandler::~AsynchIOHandler
>> (this=0xda2920, __in_chrg=<optimized out>)
>>      at /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/AsynchIOHandler.cpp:74
>> #17 0x00007fb7ca580e5f in qpid::sys::AsynchIOHandler::closedSocket
>> (this=0xda2920, s=...)
>>      at /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/AsynchIOHandler.cpp:196
>> #18 0x00007fb7ca4fa43c in qpid::sys::posix::AsynchIO::writeable
>> (this=0xebb420, h=...)
>>      at /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/posix/AsynchIO.cpp:575
>> #19 0x00007fb7ca5847a1 in operator() (a0=..., this=<optimized out>) at
>> /usr/include/boost/function/function_template.hpp:767
>> #20 qpid::sys::DispatchHandle::processEvent (this=0xebb428,
>> type=qpid::sys::Poller::WRITABLE)
>>      at /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/DispatchHandle.cpp:283
>> #21 0x00007fb7ca52258e in process (this=<synthetic pointer>,
>> this=<synthetic pointer>) at
>> /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/Poller.h:131
>> #22 qpid::sys::Poller::run (this=0xd6ff30) at
>> /usr/src/debug/qpid-cpp-0.32/src/qpid/sys/epoll/EpollPoller.cpp:522
>> #23 0x00007fb7caab8238 in qpid::broker::Broker::run (this=0xd72d90) at
>> /usr/src/debug/qpid-cpp-0.32/src/qpid/broker/Broker.cpp:522
>> #24 0x00000000004055cb in qpid::broker::QpiddBroker::execute
>> (this=this@entry=0x7fffaba3ad3e, options=<optimized out>)
>>      at /usr/src/debug/qpid-cpp-0.32/src/posix/QpiddBroker.cpp:214
>> #25 0x0000000000409004 in qpid::broker::run_broker (argc=5,
>> argv=0x7fffaba3b0d8, hidden=<optimized out>)
>>      at /usr/src/debug/qpid-cpp-0.32/src/qpidd.cpp:108
>> #26 0x00007fb7c95d4af5 in __libc_start_main (main=0x404750 <main(int,
>> char**)>, argc=5, ubp_av=0x7fffaba3b0d8, init=<optimized
>> out>,
>>      fini=<optimized out>, rtld_fini=<optimized out>,
>> stack_end=0x7fffaba3b0c8) at libc-start.c:274
>> #27 0x00000000004049c1 in _start ()
>> (gdb)
>> (gdb) fr 1
>> #1  0x00007fb7c6afc74d in pn_connection_finalize (object=0xda2c90) at
>> /usr/src/debug/qpid-proton-0.9/proton-c/src/engine/engine.c:467
>> 467      if (pn_refcount(conn) > 0) {
>> (gdb) p *conn
>> $1 = {endpoint = {type = CONNECTION, state = 18, error = 0xda3bb0,
>> condition = {name = 0xda2db0, description = 0xda2de0, info =
>> 0xda2e10},
>>      remote_condition = {name = 0xebb870, description = 0xda3180, info
>> = 0xda31d0}, endpoint_next = 0xdac6c0, endpoint_prev = 0x0,
>>      transport_next = 0x0, transport_prev = 0x0, refcount = 0,
>> modified = false, freed = true, referenced = true}, endpoint_head
>> = 0x0,
>>    endpoint_tail = 0x0, transport_head = 0x0, transport_tail = 0x0,
>> sessions = 0xda2b00, freed = 0xda4250, transport = 0x0,
>> work_head = 0x0,
>>    work_tail = 0x0, tpwork_head = 0x0, tpwork_tail = 0x0, container =
>> 0xe199b0, hostname = 0xe19a00, offered_capabilities =
>> 0xda5b50,
>>    desired_capabilities = 0xda5df0, properties = 0xda5fb0, collector =
>> 0xdac360, context = 0xda61c0, delivery_pool = 0xda6210}
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message