activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From boday <ben.o...@initekconsulting.com>
Subject Re: Fwd: Message groups enhancement feature request
Date Fri, 01 Jul 2011 14:00:49 GMT
Martin, message groups (by design) guarantee that only a single consumer will
process messages for a group at a time.  It preserves ordering of
related/group messages by single threading their processing while still
allowing concurrent processing of the queue as a whole (each group has it's
own thread/consumer)...


Martin C. wrote:
> 
> Hi,
> 
> My issue is that in a dynamically load-balancing scenario with scaling
> the number of consumers up/down, long-living consumers will tend to
> gather all available message groups and newly created won't have
> anything available. Even if a newly created consumer gets a new
> message group, it will only receive this new message group and in case
> messages within a group are rare, it is likely to be removed by
> downscaling after some idle time, as all other messages went to the
> older consumer, releasing the message group and making it more likely
> that the long-living consumer will also get one of those messages and
> gets this group assigned additionally to all the ones it already has.
> 
> I was thinking about using JMSXGroupSeq as well, but as far as I
> understand it (and the linked article), it wouldn't help me to reach
> my goal: I want to consume messages within a group one at a time,
> where I don't care which consumer consumes it, though. I want to
> ensure the following:
> 
> Producer produces Msg1, Msg2, Msg3 within Group1.
> Consumer1: handles Msg1
> Consumer1: handles Msg2
> Consumer2: handles Msg3
> 
> All I care about is, that Msg1 and Msg2 and Msg3 are not consumed in
> parallel. As far as I understood, setting JMSXGroupSeq to -1 on the
> three messages wouldn't give me this guarantee, would it? It would
> rather result in Msg1 being consumed by Consumer1 and Msg2 being
> consumed by Consumer 2 in parallel, wouldn't it?
> 
> Thanks for your input!
> 
> Best regards,
> Martin
> 
> On Fri, Jul 1, 2011 at 6:21 AM, boday
> &lt;ben.oday@initekconsulting.com&gt; wrote:
>> Is your issue that certain consumers become too slow?  If so, you can
>> always
>> set the JMSXGroupSeq header to -1 to force a new consumer (maybe set this
>> periodically or after a certain number of messages)...
>>
>> here is a good article that discusses this a bit...
>>
>> http://scottcranton.blogspot.com/2010/09/activemq-message-groups.html
>>
>> Either way, I'd think this could be a feature of the message groups.
>> Basically let you specify a frequency (either elapsed time or message
>> count)
>> to force a new consumer for a group.  Would serve to better randomize the
>> load across consumers to protect against slow consumers, etc...
>>
>>
>> Martin C. wrote:
>>>
>>> Hi,
>>>
>>> we currently use message groups to ensure ordered delivery of messages
>>> within a given group. As noted on the documentation for message
>>> groups, this often does not scale very well with variable consumer
>>> counts, as in this case the oldest consumers will hog all message
>>> groups.
>>>
>>> In our use-case, and I suppose this might apply to others as well, we
>>> simply don't care which consumer consumes which message, but only,
>>> that the next message from the same message group is not consumed
>>> before the first has been processed. So my idea would be to add a
>>> configuration option, that a consumer will release the message group
>>> directly after consuming its current message, releasing the message
>>> group for re-distribution to other brokers.
>>>
>>> Would something like this be possible / do I overlook some downside of
>>> this approach?
>>>
>>> Best regards,
>>> Martin
>>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Fwd-Message-groups-enhancement-feature-request-tp3635293p3637432.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
> 

--
View this message in context: http://activemq.2283324.n4.nabble.com/Fwd-Message-groups-enhancement-feature-request-tp3635293p3638261.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message