activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <rajdav...@gmail.com>
Subject Re: Processing multiple messages in the queue at once
Date Fri, 08 Jan 2010 07:13:32 GMT

On 7 Jan 2010, at 22:12, jongraf wrote:

>
> 1) So when onMessage() is called, it is actually coming from the  
> prefetch?
Yes - which by default is 1000
>
> 2) I'd like to take the messages from the queue and put them in a  
> collection
> and send that collection to my REST service, this way I only need to  
> make
> one HTTP call.  Do you think I'll be able to do this in my  
> MessageListener?
> Should I create a private collection in the Listener?
I'd recommend having a BlockingQueue that you fill fill with messages  
from onMessage() - and use another thread (Executor) - to pull  
messages from that Queue and batch up for your HTTP call.
>
>
> Joe Fernandez wrote:
>>
>> By default, ActiveMQ sends or streams messages to a consumer's local
>> buffer in batches. The batching of the messages can be controlled  
>> via the
>> prefetch limit. So AMQ pushes messages to the consumer as opposed to
>> having the consumer pull each message on demand.
>>
>> http://activemq.apache.org/what-is-the-prefetch-limit-for.html
>>
>> So very fast consumers would typically have high prefetch limits  
>> and slow
>> consumers low limits.
>>
>> If you set the prefetch limit to 0, the consumer will then have to  
>> pull
>> each individual message from the broker.
>>
>> AFAIK, ActiveMQ does not include a timer that, when it expires,  
>> purges a
>> queue. What about assigning an expiration time to the message?
>>
>> Joe
>> http://www.ttmsolutions.com
>>
>>
>>
>>
>> jongraf wrote:
>>>
>>> Hello fellow developers,
>>>
>>> I find myself in the midst of the age-old dilemma of needing to  
>>> provide
>>> scalable code where our load tests currently indicate that the  
>>> code is
>>> otherwise.  I am coding a message-driven POJO.  Originally I  
>>> implemented
>>> the queue using the Spring DefaultMessageListenerContainer and  
>>> discovered
>>> that this is invoked upon every message received (hence,  
>>> "listener").
>>> Because our load tests are bombarding our server, the performance
>>> benchmarks were not great.
>>>
>>> I figured that if I write my own timer instead of a listener,  
>>> performance
>>> would improve.  I coded a Quartz-triggered method that calls  
>>> iterates
>>> over jmsTemplate.recieve() for a given batch size.  The messages  
>>> are sent
>>> in one batch to a remote REST service.
>>>
>>> This has not improved performance.  After doing some research, I  
>>> found
>>> that a caveat of the jmsTemplate is that you do not really want to  
>>> be
>>> calling jmsTemplate.receive() because it opens a new connection &  
>>> session
>>> each time.  This is bad.
>>>
>>> Here is where I need your help:
>>> 1) Is there not a way to use the Spring  
>>> DefaultMessageListenerContainer
>>> to rid the queue of many messages at once?  My confusion lies in the
>>> implementation of the onMessage() method where I believe only one  
>>> message
>>> is available at a time.
>>>
>>> 2) Is there a way I can set a timer so that queue is only purged  
>>> when I
>>> want it to be?  Perhaps this goes against the MDP pattern but I  
>>> would
>>> really love some education here.
>>>
>>> The forums seem to point to JMS Transactions as a solution for  
>>> batching
>>> objects from a JMS queue.   I am also unfamiliar with how to tie  
>>> this
>>> into the DefaultMessageListenerContainer.
>>>
>>> Any help or direction you could provide would be most appreciated.
>>>
>>> Thank you,
>>>
>>> Jonathan
>>> Developer from NYC
>>>
>>
>>
>
> -- 
> View this message in context: http://old.nabble.com/Processing-multiple-messages-in-the-queue-at-once-tp27067252p27067952.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/






Mime
View raw message