activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wo ...@bill.ee>
Subject Re: Producers hang under load with AMQ 5.2
Date Wed, 02 Sep 2009 11:35:42 GMT

Hi.

We have 1 producer app and 2 consumers in total. Messages are distributed
evenly between 2 separate topics.
Slow consumer should not be a problem, both consumers use
java.util.concurrent Executors working off an unbounded queue to process the
incoming messages and every attempt has been made to make jms message
handoff to executors as fast as possible.

Messages itself are small, maybe 100 bytes of data in each. Our topic memory
limit in activemq config is set to 20mb which seems sensible enough. Would
it help to increase it further?

I'll try to confirm whether the consumer is to blame but even if it was, is
there any workaround besides not putting a load on the system? :) I'd say
its a pretty major issue with activemq, one slow consumer shouldn't be able
to bring the whole thing down.

Regards,
Tanel


Andreas Gies wrote:
> 
> Hi there,
> 
> I am wondering how many consumers there are on the topic and if there  
> are any slow consumers that
> can't keep up with the messaging load. Without having verified it, it  
> seems that your app hangs in the
> the send message waiting for the response from the broker. That would  
> happen, if the sending application
> can't place the message in the broker.
> 
> One thing strikes me odd....if all of your consumers keep consuming,  
> that shouldn't happen. So it might
> be a starting point to double check them.
> 
> As for the Advisory messages: Yes, thats normal. For more details on  
> advisories see
> http://activemq.apache.org/advisory-message.html. You *could turn  
> advisories off,
> but you need to remember to reenable them if you are switching to a  
> network of brokers
> some day ;)
> 
> Hope that helps a bit
> Andreas
> 
> 
> On Sep 2, 2009, at 7:24 AM, wo wrote:
> 
>>
>> Hi.
>>
>> We're having a problem under moderate to heavy load (~2500  
>> nonpersisted
>> messages /second on a single broker) while producers will just hang
>> indefinitely at some point and producer application needs to be  
>> restarted in
>> order to continue.
>>
>> Here's a snippet from a producer application thread dump after hang:
>>
>> at  
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java: 
>> 317)
>>        at
>> org 
>> .apache 
>> .activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>>        at
>> org 
>> .apache 
>> .activemq 
>> .transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>>        at
>> org 
>> .apache 
>> .activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java: 
>> 1195)
>>        at
>> org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1644)
>>        - locked <0x00002aaab5abeb30> (a java.lang.Object)
>>        at
>> org 
>> .apache 
>> .activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java: 
>> 227)
>>        at
>> org 
>> .apache 
>> .activemq 
>> .ActiveMQMessageProducerSupport 
>> .send(ActiveMQMessageProducerSupport.java:241)
>>        at
>> org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:597)
>>
>> So far we've tried upgrading AMQ from 5.1 to 5.2 and increasing the  
>> memory
>> limits in broker config, both giving no result, the problem still  
>> exists.
>> Neither application nor activemq log displays any errors / exceptions.
>>
>> One thing i have noticed is that for every message sent to a topic,  
>> there
>> are 2 advisory messages being produced in
>> ActiveMQ.Advisory.Producer.Topic.TOPICNAME which strikes me as odd  
>> but i
>> haven't been able to find any information in forums or manual about  
>> whether
>> this is normal behaviour?
>>
>> Producer side is a standalone Spring application with AMQ  
>> connectionfactory
>> wrapped in spring SingleConnectionFactory and used in jmstemplate,  
>> as per
>> example.
>>
>> Any insight would be greatly appreciated.
>>
>> Tanel Unt
>> Core Developer
>> Tracking Center Ltd.
>> -- 
>> View this message in context:
>> http://www.nabble.com/Producers-hang-under-load-with-AMQ-5.2-tp25251841p25251841.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Producers-hang-under-load-with-AMQ-5.2-tp25251841p25255812.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message