activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: Consumer throttling
Date Tue, 02 Feb 2016 14:30:41 GMT
The documentation you linked to sounds like Artemis allows rate-limiting
consumers individually but not rate-limiting an entire pool of consumers.
I was under the impression that Bilgin had more than one consumer and
wanted the ability to limit them as a group, but it wasn't entirely clear
so maybe I read the wrong things between the lines.

Tim
On Feb 2, 2016 6:59 AM, "Justin Bertram" <jbertram@apache.com> wrote:

> For what it's worth, I believe the ActiveMQ Artemis broker (a sub-project
> of ActiveMQ) has the feature you're looking for.  See more details in the
> documentation [1].
>
>
> Justin
>
> [1] http://activemq.apache.org/artemis/docs/1.2.0/flow-control.html
>
> ----- Original Message -----
> From: "Bilgin Ibryam" <bibryam@gmail.com>
> To: users@activemq.apache.org
> Sent: Tuesday, February 2, 2016 5:54:28 AM
> Subject: Re: Consumer throttling
>
> Thanks all for your inputs. It seems there is no solution for my
> question in ActiveMQ yet.
>
> @Christopher: I'm aware of scheduling feature, but that is more of a
> Delayer rather than Throttler. It can delay messages, but cannot
> ensure throttled consumption. If I get a burst of messages delay them,
> at a later point in time there still will be a burst of messages.
> Nothing has improved.
>
> @Tim: I have described your suggestion with two queues and Camel in
> between in as a temporary solution in my recent Camel Design Patterns
> book, but that solution is not ideal either. It is still possible to
> have messages accumulate in queueu B (for example if the consumers are
> stopped temporarily) and then have non throttled consumption breaking
> SLAs, etc.
>
> @Rau I thoughtl PFC should take care of the message backlog and
> protect the broker regardless of the reason for message accumulation,
> whether that is slow consumers, throttled consumption or consumers
> being down for some reason. Having prefetch limit of 1 is good
> suggestion though, thanks.
>
>
>
> On 30 January 2016 at 21:30, Tim Bain <tbain@alumni.duke.edu> wrote:
> > You could achieve your goal by having two queues, A and B.  Have
> producers
> > publish to A and consumers consume from B, and use an embedded Camel
> route
> > to move messages from A to B at the desired rate.
> >
> > But Raul's right: if you're going to do this, make sure there's some kind
> > of safety feature to prevent messages from backlogging on the broker.
> > Maybe you get it for free based on the characteristics of your clients,
> or
> > maybe that's some mechanism to discard built-up messages (based on time,
> > count, bytes, etc.), or a way to detect a backlog and temporarily speed
> up
> > the Camel route or to let consumers consume directly from A for a while,
> > or...
> >
> > Tim
> > On Jan 30, 2016 12:56 PM, "Raul Kripalani" <raulk@apache.org> wrote:
> >
> >> Just one remark. This could be dangerous as it could lead to an
> >> ever-growing backlog of messages in AMQ. I suppose you have already
> >> considered it, and either the producers are slower than the consumers,
> or
> >> you have multiple competing consumers, or there is a period in the day
> >> where the producers stop producing to give the consumers time to consume
> >> the existing backlog or something.
> >>
> >> If you implement this, I would reduce the prefect buffer to 1 if you
> have
> >> multiple competing consumers to achieve decent distribution and avoid
> >> starvation.
> >>
> >> Raúl.
> >> On 30 Jan 2016 18:15, "Bilgin Ibryam" <bibryam@gmail.com> wrote:
> >>
> >> > Hi all,
> >> >
> >> > Apologoes if this is a stupid question. But I wonder is there a way to
> >> > do throttling on the consumer side. Let me clarify what I mean.
> >> >
> >> > PFC can do throttling for the producer, but I need a similar behaviour
> >> > on the consumer. For example I want to configure a message consure or
> >> > message queue so that messages are consumed/released one per second.
> >> > Usually  the messages are consumed as fast as the cosnumers can take.
> >> > Then I can do some throttling in consumer Camel route, but that
> >> > throutling in Camel cannot be clustered accross instances. If the
> >> > throttling can be configured ot queue level, then I don't have to do
> >> > consumer throttling.
> >> >
> >> > Thanks,
> >> >
> >> > --
> >> > Bilgin Ibryam
> >> > Camel Committer at ASF & Integration Architect at Red Hat
> >> > Blog: http://ofbizian.com | Twitter: @bibryam
> >> >
> >> > Camel Design Patterns https://leanpub.com/camel-design-patterns
> >> > Instant Apache Camel Message Routing
> http://www.amazon.com/dp/1783283475
> >> >
> >>
>
>
>
> --
> Bilgin Ibryam
> Camel Committer at ASF & Integration Architect at Red Hat
> Blog: http://ofbizian.com | Twitter: @bibryam
>
> Camel Design Patterns https://leanpub.com/camel-design-patterns
> Instant Apache Camel Message Routing http://www.amazon.com/dp/1783283475
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message