activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tnine <todd.n...@gmail.com>
Subject Re: Input queue and output queue grouping and sequences question
Date Thu, 12 Apr 2007 17:34:45 GMT

Ok, if that's the case I seem to have completely missed the point of Message
Grouping.    It states that the following semantics are applied according to
the description in the "Impliations" section here
http://activemq.apache.org/message-groups.html. 

# partioning the traffic
# load balancing of message groups across consumers


Doesn't this mean that I can have multiple instances of my MDBs, each in its
own thread, consuming a message off of the queue?  In other words, won't
Active MQ give the next message to whatever consumer requests a message? 
So, can't I have message 1 read from the queue, and while that transaction
is processing, another node reads and begins processing message 2 BEFORE the
transaction commits for the message 1 read?  Otherwise, it becomes a
sequential consumer, which I don't want.  I need the ability to process
multiple messages concurrently across multiple nodes, but just have them
ordered in the same way on the output queue as they were when they were
inserted into the input queue.  Ordering off of the input queue is not as
important, only that they end up in order when they are put in the output
queue, which I thought I could do by copying the message sequence number.  
I guess my problem boils down to I'm having trouble distinguishing between
normal grid processing where multiple MDB instances read from a JMS queue,
and multiple MDB instances attempting to read from the same queue group in
Active MQ.  Here would be my current understanding of what you described. 
Thank you for your patience with me, I'll eventually get this through my
thick skull, if I only had a virtual white board this would be much
easier....

What I thought the documentation said.

FileReader1 -> Creates message on input queue group named "Biz1-ISODate" -->
Multiple MDB threads processing at the same time, but each MDB thread gets a
new message AFTER the previous message has been read by an MDB  (so we get
concurrent processing)-> Creates message on output queue group named
"Biz1-ISODate" --> Single threaded consumer receives messages in order.

What I understand from your post.

FileReader1 -> Creates message on input queue group named "Biz1-ISODate" -->
A single MDB thread processing one message at a time sequentially (for
single threaded processing) -> Creates message on output queue group named
"Biz1-ISODate" --> Single threaded consumer receives messages in order.

Which one is correct?

Thanks again for your help.

Todd




James.Strachan wrote:
> 
> On 4/12/07, tnine <todd.nine@gmail.com> wrote:
>>
>> Let me give a clearer example.  Lets say I have 4 "real" messages, that
>> contain data.  I have a 5th message that contains no data but closes the
>> group on the input queue.  I need to replicate the order of items from
>> the
>> input queue to the order on the output queue because the consumer of the
>> output queue needs to know when the group is finished in order to flush
>> the
>> output file to the system.  Here is the scenario I was describing
>> explained
>> a bit clearer, (I hope since I included a bit of source)
>>
>> Step 1: messages 1 through 3 are processed normally and put on the output
>> queue.
>> Step 2: message 4 begins processing, then message 5, the "close group"
>> message gets processed simultaneously.  Message 5 will most likely be
>> inserted on the output queue before message 4 since there will be no
>> processing of the closing message.
> 
> 
> So here's the cool thing about Message Groups. The contract is 1
> thread in the entire cluster of processes & threads gets the messages
> for a single message group. So you will always process message 4
> first, then message 5. So there will be no out-of-order issues. Cool
> eh? :)
> 
> -- 
> 
> James
> -------
> http://radio.weblogs.com/0112098/
> 
> 

-- 
View this message in context: http://www.nabble.com/Input-queue-and-output-queue-grouping-and-sequences-question-tf3554541s2354.html#a9964482
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message