qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Helen Kwong <helenkw...@gmail.com>
Subject Re: Java broker - message grouping in C++ compatibility mode
Date Sat, 11 Jan 2014 01:03:14 GMT
Hi Robbie,

I did some more testing to see whether message grouping will work for us,
and compared the dequeue performance of a queue using message grouping in
default Java mode, a queue using C++ mode, and a queue not using message
grouping. I found that when I use C++ mode, the performance can be much
worse than in other comparable setups, and was wondering if you could help
me understand why.

1.  In one test, I have multiple listeners to a queue, enqueue 100000
messages to it, and see how many messages are processed in 5 minutes. I
have these different setups:

- C++ mode queue with each message having a unique identifier

- C++ mode queue with all messages having no grouping identifier (so all
belong to the default group)

- default mode queue with each message having a unique grouping identifier

- default mode queue with all messages having no grouping identifier

- default mode queue with all messages having the same grouping identifier

- regular queue without a group header key configured

All setups except for the first had about 35K - 39K messages processed, but
for the first setup, there were under 8000 messages processed. What could
explain this big difference? I’ve looked at the two grouping modes’
implementations of MessageGroupManager and see that C++ mode uses
synchronized methods rather than a ConcurrentHashMap as in default mode, so
I’d guess there might be more contention because of that, but at the same
time I can’t see why, in C++ mode, having a unique identifier for each
message results in throughput that is so much worse than having all
messages in the default group.


2. I also wanted to see the impact of having many messages at the head of
the queue that a listener can’t process because the messages belong to a
group assigned to another consumer. E.g., have 10000 messages of group A,
followed by 1000 messages of group B, and listener 1 is holding on to the
first A message for a long time -- see how long it will take listener 2 to
process all the B messages. In this case C++ mode has performance that
degrades as the number of unprocessable group A messages at the front of
the queue increases, whereas default mode's performance is unaffected,
about the same as processing 1000 messages on a regular queue.

My rough guess from looking at DefinedGroupMessageGroupManager is that
whenever listener 2 is done with a group B message, the state change
listener triggers the Group.subtract() to reset pointers for other
subscriptions and consequently findEarliestAssignedAvailableEntry(). This
then has to iterate through all the group A messages before it finds the B
message. Do you think this is the reason for the results I see?

If so, is the idea here that other subscriptions of the queue could have
skipped over the messages of a group while the group was assigned to some
subscription S, so we need to tell them to set their pointers back? If that
is indeed the idea, would it be possible to optimize it such that when a
group A is assigned to S and S gets its first message of the group, we
store what that first A message / queue entry is. Then when S is done with
the last A message, we can tell other subscriptions to go back to that
first entry, without having to iterate through the queue?

Thanks a lot for your help!

Helen


On Tue, Jan 7, 2014 at 8:46 PM, Robbie Gemmell <robbie.gemmell@gmail.com>wrote:

> ...and just to be super clear, though I think it it is mentioned correctly
> in the docs this time, the 'default group' concept does not apply in the
> regular / 'non shared' grouping mode. Messages that dont specify a group
> key value in that mode are simply not grouped in any way.
>
> On 8 January 2014 04:41, Robbie Gemmell <robbie.gemmell@gmail.com> wrote:
>
> >
> > On 8 January 2014 04:33, Helen Kwong <helenkwong@gmail.com> wrote:
> >
> >> Oh I see, I thought what you meant was that I could only alter the
> default
> >> group in shared-groups mode starting with 0.24.
> >
> >
> > No, I just missed that you said 0.16 and assumed 0.24 was the version you
> > were using . You could always change it, just in more limited ways in
> > earlier releases.
> >
> > To make sure I'm
> >> understanding this correctly -- changing the the default message group
> >> name
> >> to something else in C++ mode won't change the serial processing
> behavior
> >> I
> >> saw, right?
> >
> >
> > Correct
> >
> >
> >> Messages without a group identifier will still be considered to
> >> be in a group -- rather than no group -- and they cannot be processed by
> >> multiple consumers concurrently?
> >>
> >>
> > Yes. In the C++/shared-groups mode every message is considered to be in a
> > group, it is just a case of whether the message specifies that group or
> > instead gets put into the default group.
> >
> >
> >
> >> Thanks,
> >> Helen
> >>
> >>
> >> On Tue, Jan 7, 2014 at 8:22 PM, Robbie Gemmell <
> robbie.gemmell@gmail.com
> >> >wrote:
> >>
> >> > I just noticed you said you were using 0.16, somehow glossed over it
> >> > originally and only noticed the 0.24 in the doc URL (its many hours
> past
> >> > time I was asleep, I might be getting tired).
> >> >
> >> > Realising that, I should add that prior to 0.22 the only way to alter
> >> the
> >> > default group in the shared-groups mode from 'qpid.no-group' to
> >> something
> >> > else would have been via the 'qpid.default-message-group' queue
> declare
> >> > argument when using an AMQP client to create the queue originally, and
> >> for
> >> > 0.22 itself only that and the system property approach I mentioned
> would
> >> > work.
> >> >
> >> > Robbie
> >> >
> >> > On 8 January 2014 04:03, Helen Kwong <helenkwong@gmail.com> wrote:
> >> >
> >> > > Hi Robbie,
> >> > >
> >> > > I see. Thanks for the quick response and explanation!
> >> > >
> >> > > Helen
> >> > >
> >> > >
> >> > > On Tue, Jan 7, 2014 at 7:43 PM, Robbie Gemmell <
> >> robbie.gemmell@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > Hi Helen,
> >> > > >
> >> > > > The short answer to your question is that it is the documentation
> >> which
> >> > > is
> >> > > > incorrect, and the behaviour you are seeing is expected.
> >> > > >
> >> > > > The long answer is, when that documentation was composed a segment
> >> was
> >> > > > missed out indicating this, and needs to be added to the docs.
The
> >> > > > behaviour listed for when no group is specified is only true
of
> the
> >> > > > 'non-shared' groups supported by the Java broker, in the
> C++/shared
> >> > group
> >> > > > mode any messages recieved without an explicit group value are
all
> >> > > assigned
> >> > > > to a default group of 'qpid.no-group'. This is as per the
> behaviour
> >> of
> >> > > the
> >> > > > C++ broker itself, which is explained in the C++ broker docs
at
> the
> >> end
> >> > > of
> >> > > > the following page
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> http://qpid.apache.org/releases/qpid-0.24/cpp-broker/book/Using-message-groups.html
> >> > > > .
> >> > > > For the 0.24 Java broker, this default shared group can be changed
> >> > > > broker-wide using the Java system property
> >> > > > 'qpid.broker_default-shared-message-group', or can be overriden
> for
> >> an
> >> > > > individual queue during creation programatically via AMQP clients
> or
> >> > the
> >> > > > management interfaces through use of the argument
> >> > > > 'qpid.default-message-group' or 'messageGroupDefaultGroup'.
> >> > > >
> >> > > > I coincidentally happened to have fixed a defect with the shared
> >> groups
> >> > > > functionality last night on trunk. Its not yet included in the
> >> imminent
> >> > > > 0.26 release, though I am about to request whether that is
> possible.
> >> > > > https://issues.apache.org/jira/browse/QPID-5450
> >> > > >
> >> > > > Robbie
> >> > > >
> >> > > > On 8 January 2014 02:43, Helen Kwong <helenkwong@gmail.com>
> wrote:
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > I use the Java broker and client, version 0.16, and am
> considering
> >> > > using
> >> > > > > the message grouping feature (
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://qpid.apache.org/releases/qpid-0.24/java-broker/book/Java-Broker-Queues.html#Java-Broker-Queues-OtherTypes-Message-Grouping
> >> > > > > ).
> >> > > > > From testing I've done, there seems to be a bug with the
C++
> >> > > > compatibility
> >> > > > > model, and I'm wondering if this is a known issue. Specifically,
> >> in
> >> > my
> >> > > > test
> >> > > > > I have a queue configured to use a group header field with
> >> > > > > "qpid.group_header_key" and C++ mode with
> "qpid.shared_msg_group",
> >> > and
> >> > > > have
> >> > > > > multiple listeners to the queue. Each listener will sleep
for a
> >> short
> >> > > > > amount of time when it receives a message before returning.
I
> then
> >> > > > enqueue
> >> > > > > 10 messages that do not have a value in the group header
field
> to
> >> the
> >> > > > > queue. Since the doc says that messages without a value
in the
> >> > grouping
> >> > > > > header will be delivered to any available consumer, the
> behavior I
> >> > > expect
> >> > > > > is that the messages will be processed in parallel, i.e.,
when
> >> > > listener 1
> >> > > > > is holding on to a message and sleeping, listener 2 can
receive
> >> > another
> >> > > > > message from the queue. But what I see is that the messages
are
> >> > > processed
> >> > > > > serially -- message 2 won't be received by some thread until
> >> message
> >> > 1
> >> > > is
> >> > > > > done. When I use the default mode instead of C++ mode, then
I
> get
> >> the
> >> > > > > parallel processing behavior.
> >> > > > >
> >> > > > > Is this is a known bug, and is there a fix for it already?
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Helen
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message