activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: What is the correct way to throttle ActiveMQ producers who send persistent messages in batches to a queue?
Date Thu, 03 Sep 2015 15:57:23 GMT
James,

A full description of the TCP protocol is out of scope for this mailing
list, though I highly encourage you to pick up a book about the subject
(sorry, I don't have any specific recommendations, though I'm sure the
Internet does) or read up on it at one of the various websites on the
subject.

But the brief summary is that sending and receiving on the OS's TCP stack
happens in a separate thread, so there's a buffer at the OS level on the
sending side that holds bytes received from the application (your code) but
not yet sent out on the wire, and another buffer at the OS level on the
receiving side that holds bytes received by the wire but not yet handed off
to the application (your code on the other end of the connection).  If the
receiving end stops reading bytes, the receiving buffer will eventually
fill, at which point no bytes can be sent onto the network by the sender's
TCP stack, so then its buffer starts backing up.

Once the sender's buffer is full, your send() call will block (or fail with
an exception, depending) until space frees up in the sender's buffer, which
will only happen when bytes are sent onto the network, which will only
happen when bytes are put into the receiver's buffer, which will only
happen when the receiving application accepts bytes from the network
layer.  And in this case, that will only happen when PFC is no longer
required because the limit is no longer exceeded, and that will only happen
when the consumer reads (and acks) messages from the destination.

ActiveMQ provides the ability to tune the buffer sizes of both ends of the
TCP connection via options on the tcp transport, but you need to know what
you're doing or you can really kill your performance (and that's especially
true if you have network connections with any non-trivial amount of
latency).  Since you say this is the first you're aware of TCP buffers, I'd
say you're probably a long way from being at the point where you understand
TCP well enough to safely choose the right tuning parameters, and I'd
advise you to not mess with them.  If you want to focus on changing
anything right now, I'd say it should be trying to set a
ProducerWindowSize, not a TCP buffer size.

Tim

On Tuesday, September 1, 2015, jamesTheCruncher <james@mcruncher.com> wrote:

> Hi Tim,
>
> Thank you very much for the detailed explanation.
> I agree it is not as such bad to see "Producer stopped" message immediately
> after "Producer throttled". I merely registered the observations from my
> test which had no ProducerWindowSize configured.
> Now that my doubts are cleared, let me try this and come back to you.
> I am not aware of TCP buffers that you mentioned.
> But can you tell me more about the TCP buffers of broker and producer?
> Where
> can I get more information about that? Is it possible to configure them?
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/What-is-the-correct-way-to-throttle-ActiveMQ-producers-who-send-persistent-messages-in-batches-to-a--tp4701204p4701600.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message