[ https://issues.apache.org/jira/browse/AMQCPP-454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572783#comment-13572783
]
Helen Huang edited comment on AMQCPP-454 at 2/6/13 8:33 PM:
------------------------------------------------------------
I copied the changes in InactivityMonitor.cpp from the new snapshot and started a new test.
It is still running has not caught any error yet. However, looking at the change you made,
I wonder if it alone would fix the issue. I think the problem might have been created by multiple
threads trying to access an InactivityMonitor at the same time. I have put a break point in
InactivityMonitor::stopMonitorThreads(), and found the call stack of a different thread:
activemq-cppud.dll!activemq::transport::inactivity::InactivityMonitor::stopMonitorThreads()
Line 469
activemq-cppud.dll!activemq::transport::inactivity::InactivityMonitor::close() Line 298
activemq-cppud.dll!activemq::wireformat::openwire::OpenWireFormatNegotiator::close() Line
213
activemq-cppud.dll!activemq::transport::correlator::ResponseCorrelator::close() Line 357
activemq-cppud.dll!activemq::core::ActiveMQConnection::disconnect(__int64 lastDeliveredSequenceId=0x0000000000000000)
Line 921 + 0x36 bytes
activemq-cppud.dll!activemq::core::ActiveMQConnection::close() Line 754 + 0x16 bytes
activemq-cppud.dll!activemq::cmsutil::ResourceLifecycleManager::destroy() Line 82 + 0x11
bytes
activemq-cppud.dll!activemq::cmsutil::CmsAccessor::destroy() Line 130
activemq-cppud.dll!activemq::cmsutil::CmsDestinationAccessor::destroy() Line 56
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::destroy() Line 209 + 0x8 bytes
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::ProducerExecutor::doInCms(cms::Session
* session=0x0ba2f9d8) Line 520 + 0x1a bytes
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::execute(activemq::cmsutil::SessionCallback
* action=0x012bfac4) Line 439
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::execute(activemq::cmsutil::ProducerCallback
* action=0x012bfb3c) Line 458
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::send(activemq::cmsutil::MessageCreator
* messageCreator=0x012bfc48) Line 539
CMSMessageHandler-MultiTest.exe!cmstemplate::Sender::SendMessage(std::basic_string<char,std::char_traits<char>,std::allocator<char>
> & message="114;SQSFUGJRMCOAGOHBCSXIGZAWZSHUCQSETQQVPSADEPUWLHSEITOQEJGAOSUIIAYTQXIPCZZLSBDACPYBWLIVDLPYLQDHXSVJVYHQIOKQMLFABNPIXRWHHTPZEUIJWPWTWYIACRJIDTFGDNSKAUHBWZGIETGCJLHEUDTRACBQMBZGZHVCJXEIHHOWSKDARYNZFWZVGMITKUKOFPPTLZYBBTETSKNPSAOAQIYONVHXARALFXROZGMROQYUGWBCSNPIQDZGDXQVDTYXJMBEVOHEUGTEMUGNADWUHYIAPKUNCDEHCUXNLOKDKSTUODOMYABXUMYWHSBCQFFINJBEMFQEXGBZWYCHZPUDDPMGNAMMDMTFEIBSJKCNDWLVJJCIXTFPRYLYYTGCL,
cmstemplate::ErrorCode & errorCode=CMS_SUCCESS) Line 65 + 0x2d bytes
CMSMessageHandler-MultiTest.exe!cmstemplate::TestSenderAndReceiver::run() Line 91
activemq-cppud.dll!decaf::lang::Thread::run() Line 143
activemq-cppud.dll!`anonymous namespace'::runCallback(void * arg=0x00bbb518) Line 262 + 0x11
bytes
activemq-cppud.dll!`anonymous namespace'::threadEntryMethod(void * arg=0x00bbb518) Line 250
+ 0x15 bytes
msvcr80d.dll!_callthreadstartex() Line 348 + 0xf bytes
msvcr80d.dll!_threadstartex(void * ptd=0x00bbf718) Line 331
kernel32.dll!7c80b729()
[Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]
was (Author: hhuang):
Hi Timothy,
I copied the changes in InactivityMonitor.cpp from the new snapshot and started a new test.
It is still running has not caught any error yet.
However, looking at the change you made, I wonder if it alone would fix the issue. I think
the problem might have been created by multiple threads trying to access an InactivityMonitor
at the same time. I have put a break point in InactivityMonitor::stopMonitorThreads(), and
found the call stack of a different thread:
activemq-cppud.dll!activemq::transport::inactivity::InactivityMonitor::stopMonitorThreads()
Line 469
activemq-cppud.dll!activemq::transport::inactivity::InactivityMonitor::close() Line 298
activemq-cppud.dll!activemq::wireformat::openwire::OpenWireFormatNegotiator::close() Line
213
activemq-cppud.dll!activemq::transport::correlator::ResponseCorrelator::close() Line 357
activemq-cppud.dll!activemq::core::ActiveMQConnection::disconnect(__int64 lastDeliveredSequenceId=0x0000000000000000)
Line 921 + 0x36 bytes
activemq-cppud.dll!activemq::core::ActiveMQConnection::close() Line 754 + 0x16 bytes
activemq-cppud.dll!activemq::cmsutil::ResourceLifecycleManager::destroy() Line 82 + 0x11
bytes
activemq-cppud.dll!activemq::cmsutil::CmsAccessor::destroy() Line 130
activemq-cppud.dll!activemq::cmsutil::CmsDestinationAccessor::destroy() Line 56
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::destroy() Line 209 + 0x8 bytes
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::ProducerExecutor::doInCms(cms::Session
* session=0x0ba2f9d8) Line 520 + 0x1a bytes
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::execute(activemq::cmsutil::SessionCallback
* action=0x012bfac4) Line 439
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::execute(activemq::cmsutil::ProducerCallback
* action=0x012bfb3c) Line 458
activemq-cppud.dll!activemq::cmsutil::CmsTemplate::send(activemq::cmsutil::MessageCreator
* messageCreator=0x012bfc48) Line 539
CMSMessageHandler-MultiTest.exe!cmstemplate::Sender::SendMessage(std::basic_string<char,std::char_traits<char>,std::allocator<char>
> & message="114;SQSFUGJRMCOAGOHBCSXIGZAWZSHUCQSETQQVPSADEPUWLHSEITOQEJGAOSUIIAYTQXIPCZZLSBDACPYBWLIVDLPYLQDHXSVJVYHQIOKQMLFABNPIXRWHHTPZEUIJWPWTWYIACRJIDTFGDNSKAUHBWZGIETGCJLHEUDTRACBQMBZGZHVCJXEIHHOWSKDARYNZFWZVGMITKUKOFPPTLZYBBTETSKNPSAOAQIYONVHXARALFXROZGMROQYUGWBCSNPIQDZGDXQVDTYXJMBEVOHEUGTEMUGNADWUHYIAPKUNCDEHCUXNLOKDKSTUODOMYABXUMYWHSBCQFFINJBEMFQEXGBZWYCHZPUDDPMGNAMMDMTFEIBSJKCNDWLVJJCIXTFPRYLYYTGCL,
cmstemplate::ErrorCode & errorCode=CMS_SUCCESS) Line 65 + 0x2d bytes
CMSMessageHandler-MultiTest.exe!cmstemplate::TestSenderAndReceiver::run() Line 91
activemq-cppud.dll!decaf::lang::Thread::run() Line 143
activemq-cppud.dll!`anonymous namespace'::runCallback(void * arg=0x00bbb518) Line 262 + 0x11
bytes
activemq-cppud.dll!`anonymous namespace'::threadEntryMethod(void * arg=0x00bbb518) Line 250
+ 0x15 bytes
msvcr80d.dll!_callthreadstartex() Line 348 + 0xf bytes
msvcr80d.dll!_threadstartex(void * ptd=0x00bbf718) Line 331
kernel32.dll!7c80b729()
[Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]
> Found pure function call error in CMS
> -------------------------------------
>
> Key: AMQCPP-454
> URL: https://issues.apache.org/jira/browse/AMQCPP-454
> Project: ActiveMQ C++ Client
> Issue Type: Bug
> Components: CMS Impl
> Affects Versions: 3.5.0
> Environment: Windows xp service pack 3, ActiveMQ broker 5.3.1, apr 1.4.2, apr-util
1.3.9, apr iconv 1.2.1
> Reporter: Helen Huang
> Assignee: Timothy Bish
> Priority: Critical
> Fix For: 3.5.0
>
> Attachments: cmstemplate-stress.zip, CrashHang_Report__CMSMessageHandler-MultiTest_exe__02052013182239896.mht
>
>
> WeEncountered a pure function call error in CMS during our test. The following is the
call stack:
> msvcr80d.dll!_NMSG_WRITE(int rterrnum=0x00000019) Line 198 C
> msvcr80d.dll!_purecall() Line 54 + 0x7 bytes C
> activemq-cppud.dll!decaf::util::concurrent::Lock::lock() Line 55 C++
> activemq-cppud.dll!decaf::util::concurrent::Lock::Lock(decaf::util::concurrent::Synchronizable
* object=0x0cb6de90, const bool intiallyLocked=true) Line 34 C++
> activemq-cppud.dll!activemq::transport::inactivity::InactivityMonitor::stopMonitorThreads()
Line 469 + 0x20 bytes C++
> activemq-cppud.dll!activemq::transport::inactivity::InactivityMonitor::onException(const
decaf::lang::Exception & ex={...}) Line 310 C++
> activemq-cppud.dll!activemq::transport::TransportFilter::fire(const decaf::lang::Exception
& ex={...}) Line 53 + 0x17 bytes C++
> activemq-cppud.dll!activemq::transport::TransportFilter::onException(const decaf::lang::Exception
& ex={...}) Line 46 C++
> activemq-cppud.dll!activemq::transport::IOTransport::fire(decaf::lang::Exception &
ex={...}) Line 64 + 0x17 bytes C++
> activemq-cppud.dll!activemq::transport::IOTransport::run() Line 259 C++
> activemq-cppud.dll!decaf::lang::Thread::run() Line 143 C++
> activemq-cppud.dll!`anonymous namespace'::runCallback(void * arg=0x0c85e460) Line
262 + 0x11 bytes C++
> activemq-cppud.dll!`anonymous namespace'::threadEntryMethod(void * arg=0x0c85e460)
Line 250 + 0x15 bytes C++
> msvcr80d.dll!_callthreadstartex() Line 348 + 0xf bytes C
> msvcr80d.dll!_threadstartex(void * ptd=0x1898e470) Line 331 C
> kernel32.dll!7c80b729()
> [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]
> We have 25 senders and receivers in our test, and we restart the broker every 10 seconds.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
|