activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin C." <mart...@gmx.at>
Subject Re: Fwd: Message groups enhancement feature request
Date Fri, 01 Jul 2011 07:57:36 GMT
Hi,

My issue is that in a dynamically load-balancing scenario with scaling
the number of consumers up/down, long-living consumers will tend to
gather all available message groups and newly created won't have
anything available. Even if a newly created consumer gets a new
message group, it will only receive this new message group and in case
messages within a group are rare, it is likely to be removed by
downscaling after some idle time, as all other messages went to the
older consumer, releasing the message group and making it more likely
that the long-living consumer will also get one of those messages and
gets this group assigned additionally to all the ones it already has.

I was thinking about using JMSXGroupSeq as well, but as far as I
understand it (and the linked article), it wouldn't help me to reach
my goal: I want to consume messages within a group one at a time,
where I don't care which consumer consumes it, though. I want to
ensure the following:

Producer produces Msg1, Msg2, Msg3 within Group1.
Consumer1: handles Msg1
Consumer1: handles Msg2
Consumer2: handles Msg3

All I care about is, that Msg1 and Msg2 and Msg3 are not consumed in
parallel. As far as I understood, setting JMSXGroupSeq to -1 on the
three messages wouldn't give me this guarantee, would it? It would
rather result in Msg1 being consumed by Consumer1 and Msg2 being
consumed by Consumer 2 in parallel, wouldn't it?

Thanks for your input!

Best regards,
Martin

On Fri, Jul 1, 2011 at 6:21 AM, boday <ben.oday@initekconsulting.com> wrote:
> Is your issue that certain consumers become too slow?  If so, you can always
> set the JMSXGroupSeq header to -1 to force a new consumer (maybe set this
> periodically or after a certain number of messages)...
>
> here is a good article that discusses this a bit...
>
> http://scottcranton.blogspot.com/2010/09/activemq-message-groups.html
>
> Either way, I'd think this could be a feature of the message groups.
> Basically let you specify a frequency (either elapsed time or message count)
> to force a new consumer for a group.  Would serve to better randomize the
> load across consumers to protect against slow consumers, etc...
>
>
> Martin C. wrote:
>>
>> Hi,
>>
>> we currently use message groups to ensure ordered delivery of messages
>> within a given group. As noted on the documentation for message
>> groups, this often does not scale very well with variable consumer
>> counts, as in this case the oldest consumers will hog all message
>> groups.
>>
>> In our use-case, and I suppose this might apply to others as well, we
>> simply don't care which consumer consumes which message, but only,
>> that the next message from the same message group is not consumed
>> before the first has been processed. So my idea would be to add a
>> configuration option, that a consumer will release the message group
>> directly after consuming its current message, releasing the message
>> group for re-distribution to other brokers.
>>
>> Would something like this be possible / do I overlook some downside of
>> this approach?
>>
>> Best regards,
>> Martin
>>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Fwd-Message-groups-enhancement-feature-request-tp3635293p3637432.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
View raw message