activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: What is the correct way to throttle ActiveMQ producers who send persistent messages in batches to a queue?
Date Wed, 19 Aug 2015 04:48:25 GMT
I think you might be misunderstanding the (not very clearly written)
content on the PFC wiki page.  (Either that, or my understanding of PFC is
wrong.)  As I understand it, the references to differences in behavior for
async senders on that page are not between performing and not performing
PFC, but between performing per-producer and whole-connection PFC.  When a
synchronous sender sends a message, it must receive an acknowledgement of
receipt from the broker before sending the next message; when a message is
received that pushes the store beyond some limit, the broker simply doesn't
tell the producer that it can send another until space becomes available,
stopping the producer while leaving the connection available for other
producers on other destinations.  On the other hand, when an asynchronous
sender sends a message, it can immediately send another message (or a
hundred more); if the receipt of a message pushes the store beyond some
limit, there is nothing to prevent the producer from putting another
message onto the socket.  When the broker receives that message that it
can't process, it has nothing to do but stop processing all messages on
that connection (causing the messages to queue up in the receive buffer and
then eventually in the producer's send buffer), so it PFCs all producers
that are sharing that connection.  The result is that with an asynchronous
producer, PFC can affect producers whose messages would not otherwise be
subject to PFC.  So although it sounds like you were expecting that PFC
would not happen for async sends unless the producer window size is set, I
don't believe that is the case and I think you'll simply see different
collateral effects when it does kick in.

None of that is to imply that PFC shouldn't work for all batch sizes
(including 1), just that I wouldn't expect the behavior you're seeing to be
because of asynchronous sends (and so I don't think that setting a producer
window size should be required to fix the behavior).  And certainly you
should be able to test that hypothesis by setting the jms.useAsyncSend=true
URI option with a batch size of 1; if that doesn't change the behavior of
your test, then it's not asynchronous sends per se that are at fault.

With that being said, I've never sent messages in batches (nor researched
how to do so), so there could easily be something fundamentally
incompatible between batches > 1 and PFC that I'm not aware of, so
hopefully someone with more knowledge of batch sends will weigh in.

Tim

On Tue, Aug 18, 2015 at 7:50 PM, jamesTheCruncher <james@mcruncher.com>
wrote:

> I have a producer which sends persistent messages in /batches/ to a queue
> leveraging JMS transaction.
>
> I have tested and found that /Producer Flow Control/ is applied when using
> a
> batch size of 1. I could see my producer being throttled as per the memory
> limit I have configured for the queue. Here's my Producer Flow Control
> configuration:
>
>     <policyEntry queue="foo" optimizedDispatch="true"
>          producerFlowControl="true" memoryLimit="1mb">
>     </policyEntry>
>
> The number of pending messages in the queue are in control which I see as
> the evidence for /Producer Flow Control/ in action.
>
> However, when the batch size is increased to 2, I found that this memory
> limit is not respected and the producer is NOT THROTTLED at all. The
> evidence being the number of pending messages in the queue continue to
> increase till it hits the /storeUsage/ limit configured.
>
> I understand this might be because the messages are sent in asynchronous
> fashion when the batch size is more than 1 even though I haven't explicitly
> set /useAsyncSend/ to /true/.
>
> ActiveMQ's  Producer Flow Control documentation
> <http://activemq.apache.org/producer-flow-control.html>   mentions that to
> throttle asynchronous publishers, we need to configure /Producer Window
> Size/ in the producer which shall force the Producer to wait for
> acknowledgement once the window limit is reached.
>
> However, when I configured /Producer Window Size/ in my producer and
> attempted to send messages in batches, an exception is thrown and no
> messages were sent.
>
> This makes me think and ask this question, "Is it possible to configure
> /Producer Window Size/ while sending persistent messages in /batches/?".
>
> If not, then what is the correct way to throttle the producers who send
> persistent messages in /batches/?
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/What-is-the-correct-way-to-throttle-ActiveMQ-producers-who-send-persistent-messages-in-batches-to-a--tp4701204.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message