qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Godfrey <rob.j.godf...@gmail.com>
Subject Re: all subscribers not equal?
Date Wed, 27 Jan 2010 21:22:47 GMT
Hi Mark,

I can give an outline of how the Java Broker distributes messages
between subscriptions... I'm not familiar with anything the Python
client may do...

In general, when there is no backlog in the queue, the Java Broker
will round-robin between subscriptions which have available credit.

If a subscription runs out of credit then it is marked as suspended.
When such a subscription gets credit again, or when a new subscription
is added to the queue then the queue will attempt to send as many
messages as it can to this new (or unsuspended) subscription.  (To be
absolutely accurate it attempts to send up to 10 messages, then yields
- but schedules another attempt to send 10 messages, and so on).

I'm not sure if the above totally explains what you are seeing -
certainly I don't see why the subscriptions should fail to keep up
with the publisher (until you kill the first subscriber) in the way
you describe.

If you could provide a simple example that shows the same behaviour,
that would be fantastic...  In the meantime I will have a more
thorough dive into the broker code to see if I can spot anything

Hope this helps,

2010/1/27 mARK bLOORE <mbloore@gmail.com>:
> I am using the java broker 0.5 and the python client.  I have a queue
> with one publisher and (usually) three subscribers.
> The publisher sends messages at a fairly constant rate.
> The subscribers take messages in a single thread at an unlimited rate,
> put them in a buffer, and ack the messages immediately.  If the buffer
> fills then they block before the ack.
> Normally all the subscribers get messages at about the same rate, and
> the queue's message count is mostly zero.  If a subscriber starts to
> block then the message count may rise, and when it gets to the tens of
> thousands I reduce the publication rate.  But an odd situation
> appears:  The blocked subscriber takes messages into its buffer as
> fast as it takes them out, but the other two subscribers get messages
> at only one third of the rate that the publisher is adding them.  That
> first subscriber is running many times faster.
> If I add a fourth subscriber it gets messages at one quarter the
> publication rate, and the other two start getting messages at that
> rate too.  If I kill the fourth subscriber rates return to what they
> were.
> If I kill the first subscriber then the other two start taking
> messages very fast, and the backlog in the queue quickly disappears.
> It seems as if the broker has earmarked the backlogged messages for
> that one subscriber, and won't deliver them to any other unless that
> one goes away.  Could that be the case?  Note that a subscriber may
> have at most one unacked message.
> I'm afraid I can't abstract any reasonable amount of code to display.
> I'm not sure that would help anyway.

> --
> mARK bLOORE <mbloore@gmail.com>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org

Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

View raw message