activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: NMSXGroupID problem - two different consumers received message for one group
Date Fri, 18 Dec 2015 18:16:41 GMT
I don't have any insider knowledge about whether a message group is closed
when the broker receives the message or when it becomes a candidate for
being dispatched, though other people on this list may and may be able to
speak more authoritatively to how it's done.

With that said, the behavior you're seeing seems to indicate that the
message group is closed when the message is received, which seems to be
confirmed by https://issues.apache.org/jira/browse/AMQ-3027 and by Tamas
Cserveny's comment on https://issues.apache.org/jira/browse/AMQ-2106.

Hopefully we can get AMQ-3027 fixed in a future version.  (Add your vote to
the issue if you want it fixed.)  Till then, if your messages are all
persistent, you could periodically restart the broker; if you're lucky, all
the consumers will reconnect at just about the same time, and you'll get
even distribution across them.  (You could help your odds here by using the
consumersBeforeDispatchStarts and timeBeforeDispatchStarts policyEntry
attributes as described in http://activemq.apache.org/message-groups.html
to help make sure you don't start dispatching messages with just one
consumer connected.)

If some of your messages are non-persistent and you're not willing to lose
them in order to achieve rebalancing, you're probably out of luck; either
do what you're doing now (and deal with concurrent consumption), or don't
close the group and just deal with the uneven distribution when it occurs.
Disconnecting all the consumers probably won't help, because it becomes a
race condition for how many of them are connected when the broker starts
assigning message groups; if they don't all reconnect at essentially the
same time, the broker will just hand them all to the first one, and you'll
have even worse distribution than you did before.

Tim

On Thu, Dec 17, 2015 at 7:42 AM, sburczymucha <sburczymuchatefek@gmail.com>
wrote:

> Tim Bain wrote
> > Are all messages being immediately consumed, or is there a backlog of
> > messages for a given message group when you close the message group?  I
> > would expect the message group to be closed immediately when the message
> > with NMSXGroupSeg=-1 is sent (not when it is consumed), so if there is a
> > backlog I would expect the new consumer to start processing with the next
> > undispatched message (which would be in the 200s), even if the old
> > consumer
> > is still processing its current message plus the one in its prefetch
> > buffer.
>
> Yes, there are a backlog messages even after a group is closed. I thought
> that until a group is not closed (until message with NMSXGroupSeg=-1 is not
> processed and committed) it belongs to only one consumer.
>
> By the way, does NMSXGroupSeg needs to be in order or the NMSXGroupSeg
> order
> does not have any impact on processing? What about the situation when
> current NMSXGroupSeg has max value and next one has 1 (starting from
> scratch)?
>
>
> Tim Bain wrote
> > Why are you closing the message group?  Seems like you could avoid the
> > whole problem just by doing that...
>
> What I'm trying to achieve is to periodically rebalance message groups
> along
> currently connected consumers (number of consumers can vary from time to
> time). I want to avoid the situation when new consumers are idle because
> all
> groups are processed by other consumers. I also need to keep the order of
> messages within group. Is there a better way to do it? Disconnect all
> consumers and reconnect?
>
> Thank you,
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/NMSXGroupID-problem-two-different-consumers-received-message-for-one-group-tp4705096p4705111.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message