activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Strachan <>
Subject Re: Fwd: Message groups enhancement feature request
Date Fri, 01 Jul 2011 14:09:08 GMT
So it sounds like the current Message Groups behaviour is fine; its
just you want to ensure fair load balancing of the message groups
across consumers right? So if lots of other clients are restarted; the
long running clients could end up claiming too many of the message

I wonder if just recycling consumers every now and again would do the
trick? i.e. a pure client side thing to terminate a consumer every now
and again to force rebalancing of message groups?

(Getting the broker to automatically recycle consumers would be ideal;
but then it'd have to know that a consumer has no pending acks in a
particular message group to avoid the issue of parallel consumption in
a single message group).

On 1 July 2011 08:57, Martin C. <> wrote:
> Hi,
> My issue is that in a dynamically load-balancing scenario with scaling
> the number of consumers up/down, long-living consumers will tend to
> gather all available message groups and newly created won't have
> anything available. Even if a newly created consumer gets a new
> message group, it will only receive this new message group and in case
> messages within a group are rare, it is likely to be removed by
> downscaling after some idle time, as all other messages went to the
> older consumer, releasing the message group and making it more likely
> that the long-living consumer will also get one of those messages and
> gets this group assigned additionally to all the ones it already has.
> I was thinking about using JMSXGroupSeq as well, but as far as I
> understand it (and the linked article), it wouldn't help me to reach
> my goal: I want to consume messages within a group one at a time,
> where I don't care which consumer consumes it, though. I want to
> ensure the following:
> Producer produces Msg1, Msg2, Msg3 within Group1.
> Consumer1: handles Msg1
> Consumer1: handles Msg2
> Consumer2: handles Msg3
> All I care about is, that Msg1 and Msg2 and Msg3 are not consumed in
> parallel. As far as I understood, setting JMSXGroupSeq to -1 on the
> three messages wouldn't give me this guarantee, would it? It would
> rather result in Msg1 being consumed by Consumer1 and Msg2 being
> consumed by Consumer 2 in parallel, wouldn't it?
> Thanks for your input!
> Best regards,
> Martin
> On Fri, Jul 1, 2011 at 6:21 AM, boday <> wrote:
>> Is your issue that certain consumers become too slow?  If so, you can always
>> set the JMSXGroupSeq header to -1 to force a new consumer (maybe set this
>> periodically or after a certain number of messages)...
>> here is a good article that discusses this a bit...
>> Either way, I'd think this could be a feature of the message groups.
>> Basically let you specify a frequency (either elapsed time or message count)
>> to force a new consumer for a group.  Would serve to better randomize the
>> load across consumers to protect against slow consumers, etc...
>> Martin C. wrote:
>>> Hi,
>>> we currently use message groups to ensure ordered delivery of messages
>>> within a given group. As noted on the documentation for message
>>> groups, this often does not scale very well with variable consumer
>>> counts, as in this case the oldest consumers will hog all message
>>> groups.
>>> In our use-case, and I suppose this might apply to others as well, we
>>> simply don't care which consumer consumes which message, but only,
>>> that the next message from the same message group is not consumed
>>> before the first has been processed. So my idea would be to add a
>>> configuration option, that a consumer will release the message group
>>> directly after consuming its current message, releasing the message
>>> group for re-distribution to other brokers.
>>> Would something like this be possible / do I overlook some downside of
>>> this approach?
>>> Best regards,
>>> Martin
>> --
>> View this message in context:
>> Sent from the ActiveMQ - User mailing list archive at

Twitter: jstrachan, fusenews

Open Source Integration and Messaging

View raw message