activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gerald Kaas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQCPP-277) Freeze when creating multiple Consumers
Date Tue, 04 May 2010 20:44:34 GMT

    [ https://issues.apache.org/activemq/browse/AMQCPP-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59190#action_59190
] 

Gerald Kaas commented on AMQCPP-277:
------------------------------------

I am also having problems with ActiveMQ-CPP freezing on Windows when calling setMessageListener.
I have been spending a day or so trying to track it down and there seems to be a race-condition
deadlock occurring within the Thread::join() function. You are not probably seeing this with
pthreads since you are just calling pthread_join. On the Windows platform you are calling
Thread::join( INFINITE, 0 ). The later uses a mutex to try to determine when a thread is dead
but as far as I can tell, the thread is still terminating and thus some other non-mutex state
variables are getting mixed up. I haven't tried but I bet if you change Thread::join() to
use Thread::join( INFINITE, 0 ) for both Windows and Linux, you will see the problems on both
platforms.

I was able to reproduce the problem very reliably by subscribing to various topics in a loop.
It would ALWAYS freeze in ActiveMQSessionExecutor::stop -> taskRunner->shutdown ->
this->thread->join. You may need a multiple core machine to see the problem (I have
4 cores). The DedicatedTaskRunner thread would already be terminated but the join would block.

I fixed the problem by replacing

Thread::join() ...
{
...
  Thread::join ( INFINITE, 0 );
...
}

to

{
...
  WaitForSingleObject(properties->handle, INFINITE);
...
}

to match the same logic in Linux.

Your threadWorker function may also be broken on Windows. On LINUX you called thread_detach
which doesn't actually terminate the thread and then sets the state variable. On Windows you
go ahead an terminate the thread and this set the state variable. You might want to move that
assignment.

Please let me know what you find out. As far as I know, this bug hasn't been fixed in the
trunk and is very reliable. 

> Freeze when creating multiple Consumers
> ---------------------------------------
>
>                 Key: AMQCPP-277
>                 URL: https://issues.apache.org/activemq/browse/AMQCPP-277
>             Project: ActiveMQ C++ Client
>          Issue Type: Bug
>    Affects Versions: 3.1
>         Environment: Windows 7 x64, ActiveMQ Server 5.3, Java 6.0.17, Visual Studio 2008
SP1, Windows 7 SDK, APR 1.3.9
>            Reporter: Eddie Fast
>            Assignee: Timothy Bish
>             Fix For: 3.2.0
>
>
> We create multiple MessageConsumers with different selectors.  This can be upwards of
50 or so consumers per client.
> We recently upgraded to 3.1 from 2.2.2.    We also upgraded our ActiveMQ Server to 5.3
from 5.2.
> After upgrading, we've been seeing random, but frequent freezing in our clients.  I've
narrowed it down to where it creates the MessageConsumer and calls setMessageListener();
> It seems to be stuck, waiting for a thread to join.
> I've modified the main.cpp to exhibit this behavior, but I've only been able to reproduce
it randomly.   It seems to happen more frequently if you stick a breakpoint on this line in
Thread.cpp,  Thread::join():
> Thread::join( INFINITE, 0 );
> Here's the callstack where it freezes, which is consistent in our clients as well as
the main.cpp example:
>  	ntdll.dll!76fff871() 	
>  	[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]	
>  	ntdll.dll!76fff871() 	
>  	KernelBase.dll!75e20816() 	
>  	kernel32.dll!755b1138() 	
>  	activemq-cppd.dll!decaf::internal::util::concurrent::ConditionImpl::wait(decaf::util::concurrent::ConditionHandle
* condition=0x0239e0e0, __int64 mills=4294967295, __int64 nanos=0)  Line 110 + 0x10 bytes
C++
>  	activemq-cppd.dll!decaf::util::concurrent::Mutex::wait(__int64 millisecs=4294967295,
int nanos=0)  Line 124 + 0x20 bytes	C++
>  	activemq-cppd.dll!decaf::lang::Thread::join(__int64 millisecs=4294967295, unsigned
int nanos=0)  Line 464 + 0x36 bytes	C++
>  	activemq-cppd.dll!decaf::lang::Thread::join()  Line 421	C++
> >	activemq-cppd.dll!activemq::threads::DedicatedTaskRunner::shutdown()  Line 83	C++
>  	activemq-cppd.dll!activemq::core::ActiveMQSessionExecutor::stop()  Line 110	C++
>  	activemq-cppd.dll!activemq::core::ActiveMQSession::stop()  Line 807	C++
>  	activemq-cppd.dll!activemq::core::ActiveMQConsumer::setMessageListener(cms::MessageListener
* listener=0x0018fe0c)  Line 523	C++
>  	vs2005-activemq-example.exe!HelloWorldConsumer::run()  Line 247 + 0x5b bytes	C++
>  	activemq-cppd.dll!decaf::lang::ThreadProperties::runCallback(decaf::lang::ThreadProperties
* properties=0x01df7f68)  Line 133 + 0x11 bytes	C++
>  	activemq-cppd.dll!`anonymous namespace'::threadWorker(void * arg=0x01df7f68)  Line
204 + 0x9 bytes	C++
>  	msvcr90d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
>  	msvcr90d.dll!_threadstartex(void * ptd=0x01dfafd8)  Line 331	C
> To modify main.cpp, it's pretty simple.  Instead of creating 1 MessageConsumer in HelloWorldConsumer::run(),
I've created 50:
> {quote}
> // put this line with the class variables
> std::vector<MessageConsumer*> consumers;
> for ( int i = 0; i < 50; i++ )
> {{
>          // Create a MessageConsumer from the Session to the Topic or Queue
>          consumers.push_back( session->createConsumer( destination ) );
>          consumers.back()->setMessageListener( this );
> }}
> {quote}
> I'm using the standard .conf for the server.    I've reverted to 2.2.2 and have no issues.
> Any ideas?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message