activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Nicholson <robert.nichol...@gmail.com>
Subject Re: what is best practices to implement Multithread + JMS ?
Date Tue, 22 Sep 2009 13:42:12 GMT
I know of no way to only acknowledge what you've processed without  
potentially acknowledging what you're might not have already processed.

In order to ensure that all records I have received are processed I  
currently store the state in the database such that if my process  
needs to be
restarted without having processed what's been removed from the queue  
it can consult the database to determine what's left to do.

This obviously is a very expensive operation and I'm looking for ideas  
as to how to avoid this?

One way I found was to not do the processing in a separate thread. IF  
I dispatch to "child queues" I can then have a transacted session and
have consumers/receivers attached to those queues such that I know  
when I've committed the session I've processed what was received. I  
believe
it's still possible to commit messages that weren't processed but this  
seems to have the semantics where if I rolled back the session the  
messages
are still left on the queue.

When you introduce a thread pool and do the work in worker threads it  
then becomes challenging to get the right semantics where you know
that if something hasn't been processed but a delivery attempt was  
made that that will still be on the queue. JMS doesn't let me do  
anything
with the Session in a worker thread except close it. I cannot commit  
or acknowledge in a worker thread. ie. the place where I know I've fully
processed the message or not.

Looking for ideas.

On Sep 20, 2009, at 12:11 PM, Bruce Snyder wrote:

> On Fri, Sep 18, 2009 at 8:42 PM, Robert Nicholson
> <robert.nicholson@gmail.com> wrote:
>> I've been looking at DMLC lately.
>>
>> Acknowledge semantics are a little different when using it though?
>
> There are four types of acknowledgement in the message listener
> container as listed in the AbstractMessageListenerContainer. I'd
> copy/paste the acknowledgement modes here but the Javadoc is kinda big
> but it's comprehensive. Take a look here:
>
> http://static.springsource.org/spring/docs/2.5.x/api/org/springframework/jms/listener/AbstractMessageListenerContainer.html
>
>> Isn't there the notion that you acknowledge before dispatch and  
>> therefore
>> there
>> lies an opportunity to acknowledge something from the queue and not  
>> actually
>> process it
>> because of an outage.
>
> Acknowledgement semantics depend on the mode you choose. Here are some
> items that tend to confuse:
>
> 1) "sessionAcknowledgeMode" set to "AUTO_ACKNOWLEDGE" (default):
> Automatic message acknowledgment before listener execution; no
> redelivery in case of exception thrown.
>
> 2) "sessionAcknowledgeMode" set to "CLIENT_ACKNOWLEDGE": Automatic
> message acknowledgment after successful listener execution; no
> redelivery in case of exception thrown.
>
> 3) "sessionAcknowledgeMode" set to "DUPS_OK_ACKNOWLEDGE": Lazy message
> acknowledgment during or after listener execution; potential
> redelivery in case of exception thrown.
>
> 4) "sessionTransacted" set to "true": Transactional acknowledgment
> after successful listener execution; guaranteed redelivery in case of
> exception thrown.
>
>> JMS doesn't let you commit the session in any other thread than the  
>> thread
>> of control
>> of the session so you aren't going to have a guarantee that you  
>> actually
>> processed
>> what was acknowledged/committed in the Session.
>
> Again, that depends on the ack mode. See above.
>
>> Also, DMLC doesn't do anything for you regarding order of  
>> processing so how
>> does
>> Spring address that problem? Am I expected to have N's DMLC's and  
>> dispatch
>> based
>> on my own rules rather than something that can share from a common  
>> thread
>> pool but
>> still guarantee some order of processing?
>
> Controlling the order in which messages are processed on the
> client-side is not a goal of the Spring message listener container
> architecture. In a scenario where the order of processing is
> important, you would only want to have a single consumer. Either that
> or you'd need to wholly separate the consumption/ack stage from the
> processing stage. Still, you'd need some additional strategy for such
> a restriction. Please explain the use case you have in mind for
> controlling the order in which messages are processed.
>
> Bruce
> -- 
> perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\! 
> G;6%I;\"YC;VT*"
> );'
>
> ActiveMQ in Action: http://bit.ly/2je6cQ
> Blog: http://bruceblog.org/
> Twitter: http://twitter.com/brucesnyder


Mime
View raw message