Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Received-SPF: pass (nike.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <28451485.post@talk.nabble.com>
Date: Tue, 4 May 2010 11:36:02 -0700 (PDT)
From: blitzkriegoutlaw <blitzkriegoutlaw@hotmail.com>
To: dev@activemq.apache.org
Subject: Re: [jira] Commented: (AMQCPP-277) Freeze when creating multiple
 Consumers
In-Reply-To: <353933691.8601268085225095.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <403595662.5741263273857117.JavaMail.jira@brutus.apache.org>
 <353933691.8601268085225095.JavaMail.jira@brutus.apache.org>


Issue Tracker seems to be down so I am going to try to respond this way. I am
also having problems with ActiveMQ-CPP freezing on Windows when calling
setMessageListener. I have been spending a day or so trying to track it down
and there seems to be a race-condition deadlock occurring within the
Thread::join() function. You are not probably seeing this with pthreads
since you are just calling pthread_join. On the Windows platform you are
calling Thread::join( INFINITE, 0 ). The later uses a mutex to try to
determine when a thread is dead but as far as I can tell, the thread is
still terminating and thus some other non-mutex state variables are getting
mixed up. I haven't tried but I bet if you change Thread::join() to use
Thread::join( INFINITE, 0 ) for both Windows and Linux, you will see the
problems on both platforms.

I was able to reproduce the problem very reliably by subscribing to various
topics in a loop. It would ALWAYS freeze in ActiveMQSessionExecutor::stop ->
taskRunner->shutdown -> this->thread->join. The DedicatedTaskRunner thread
would already be terminated but the join would come back.

I fixed the problem by replacing

Thread::join() ...
{
...
  Thread::join ( INFINITE, 0 );
...
}

to 

{
...
  WaitForSingleObject(properties->handle, INFINITE);
...
}

to match the same logic in Linux.

Your threadWorker function may also be broken on Windows. On LINUX you
called thread_detach which doesn't actually terminate the thread and then
sets the state variable. On Windows you go ahead an terminate the thread and
this set the state variable. You might want to move that assignment.

Please let me know what you find out. As far as I know, this bug hasn't been
fixed in the trunk and is very reliable.


JIRA jira@apache.org wrote:
> 
> 
>     [
> https://issues.apache.org/activemq/browse/AMQCPP-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=58099#action_58099
> ] 
> 
> Timothy Bish commented on AMQCPP-277:
> -------------------------------------
> 
> I'm still unable to reproduce this on my Windows machines.
> 
>> Freeze when creating multiple Consumers
>> ---------------------------------------
>>
>>                 Key: AMQCPP-277
>>                 URL: https://issues.apache.org/activemq/browse/AMQCPP-277
>>             Project: ActiveMQ C++ Client
>>          Issue Type: Bug
>>    Affects Versions: 3.1
>>         Environment: Windows 7 x64, ActiveMQ Server 5.3, Java 6.0.17,
>> Visual Studio 2008 SP1, Windows 7 SDK, APR 1.3.9
>>            Reporter: Eddie Fast
>>            Assignee: Timothy Bish
>>             Fix For: 3.2.0
>>
>>
>> We create multiple MessageConsumers with different selectors.  This can
>> be upwards of 50 or so consumers per client.
>> We recently upgraded to 3.1 from 2.2.2.    We also upgraded our ActiveMQ
>> Server to 5.3 from 5.2.
>> After upgrading, we've been seeing random, but frequent freezing in our
>> clients.  I've narrowed it down to where it creates the MessageConsumer
>> and calls setMessageListener();
>> It seems to be stuck, waiting for a thread to join.
>> I've modified the main.cpp to exhibit this behavior, but I've only been
>> able to reproduce it randomly.   It seems to happen more frequently if
>> you stick a breakpoint on this line in Thread.cpp,  Thread::join():
>> Thread::join( INFINITE, 0 );
>> Here's the callstack where it freezes, which is consistent in our clients
>> as well as the main.cpp example:
>>  	ntdll.dll!76fff871() 	
>>  	[Frames below may be incorrect and/or missing, no symbols loaded for
>> ntdll.dll]	
>>  	ntdll.dll!76fff871() 	
>>  	KernelBase.dll!75e20816() 	
>>  	kernel32.dll!755b1138() 	
>>  
>> activemq-cppd.dll!decaf::internal::util::concurrent::ConditionImpl::wait(decaf::util::concurrent::ConditionHandle
>> * condition=0x0239e0e0, __int64 mills=4294967295, __int64 nanos=0)  Line
>> 110 + 0x10 bytes	C++
>>  	activemq-cppd.dll!decaf::util::concurrent::Mutex::wait(__int64
>> millisecs=4294967295, int nanos=0)  Line 124 + 0x20 bytes	C++
>>  	activemq-cppd.dll!decaf::lang::Thread::join(__int64
>> millisecs=4294967295, unsigned int nanos=0)  Line 464 + 0x36 bytes	C++
>>  	activemq-cppd.dll!decaf::lang::Thread::join()  Line 421	C++
>> >	activemq-cppd.dll!activemq::threads::DedicatedTaskRunner::shutdown() 
>> Line 83	C++
>>  	activemq-cppd.dll!activemq::core::ActiveMQSessionExecutor::stop()  Line
>> 110	C++
>>  	activemq-cppd.dll!activemq::core::ActiveMQSession::stop()  Line 807	C++
>>  
>> activemq-cppd.dll!activemq::core::ActiveMQConsumer::setMessageListener(cms::MessageListener
>> * listener=0x0018fe0c)  Line 523	C++
>>  	vs2005-activemq-example.exe!HelloWorldConsumer::run()  Line 247 + 0x5b
>> bytes	C++
>>  
>> activemq-cppd.dll!decaf::lang::ThreadProperties::runCallback(decaf::lang::ThreadProperties
>> * properties=0x01df7f68)  Line 133 + 0x11 bytes	C++
>>  	activemq-cppd.dll!`anonymous namespace'::threadWorker(void *
>> arg=0x01df7f68)  Line 204 + 0x9 bytes	C++
>>  	msvcr90d.dll!_callthreadstartex()  Line 348 + 0xf bytes	C
>>  	msvcr90d.dll!_threadstartex(void * ptd=0x01dfafd8)  Line 331	C
>> To modify main.cpp, it's pretty simple.  Instead of creating 1
>> MessageConsumer in HelloWorldConsumer::run(), I've created 50:
>> {quote}
>> // put this line with the class variables
>> std::vector<MessageConsumer*> consumers;
>> for ( int i = 0; i < 50; i++ )
>> {{
>>          // Create a MessageConsumer from the Session to the Topic or
>> Queue
>>          consumers.push_back( session->createConsumer( destination ) );
>>          consumers.back()->setMessageListener( this );
>> }}
>> {quote}
>> I'm using the standard .conf for the server.    I've reverted to 2.2.2
>> and have no issues.
>> Any ideas?
> 
> -- 
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/-jira--Created%3A-%28AMQCPP-277%29-Freeze-when-creating-multiple-Consumers-tp27122840p28451485.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.