activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <christian.po...@gmail.com>
Subject Re: Memory limit errors
Date Wed, 02 Jan 2013 21:04:46 GMT
So persistent=true is a broker setting. The setting in question for your
scenario is that for the client (producer) of the message.

See here:

http://activemq.apache.org/what-is-the-difference-between-persistent-and-non-persistent-delivery.html


On Wed, Jan 2, 2013 at 2:02 PM, Mohit Anchlia <mohitanchlia@gmail.com>wrote:

> Thanks for the info. I'll look at those parameter.
>
> It does look like that I am using persistence based on below
> "persistent=true" parameter. So can I safely assume that only one producer
> gets blocked?
>
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="static-broker1" persistent="true"
> dataDirectory="${activemq.data}">
>
> On Wed, Jan 2, 2013 at 12:53 PM, Christian Posta
> <christian.posta@gmail.com>wrote:
>
> > Yah, it's probably best to figure out how things are being sent right
> now.
> >
> > If messages are sent async and without a producer window then you'll hit
> > the type of flow control that you did.
> >
> > Messages are sent async by default if using non-persistent messages or
> > during a transaction. The only time any synchronous communication will be
> > done during a transaction is when you commit (or rollback).
> >
> > So you can set ActiveMQConnection#setAlwaysSyncSend which will always
> send
> > sync for all types of sends (non-persistent and during a transaction).
> This
> > will enable the type of flow control on the broker side that blocks only
> > the one producer. The drawbacks are it has more network overhead and is
> > likely slower. How much slower is something you may wish to measure.
> > Depending on your application requirements, it might not be a big deal.
> >
> > You can also set the producer window on the client. The producer window
> > establishes a client-side contract that basically says "I will only sned
> N
> > number of bytes to you until I hear back that the message has been
> > received" You can set this value (in number of bytes) on
> > ActiveMQConnection#setProducerWindowSize. This will also achieve blocking
> > only the one producer instead of the entire connection.
> >
> > Lastly, note that persistent messages are always sent synchronous so by
> > default would block only the producer and not the entire connection.
> >
> >
> > On Wed, Jan 2, 2013 at 1:40 PM, Mohit Anchlia <mohitanchlia@gmail.com
> > >wrote:
> >
> > > Thanks! Sounds like there might be another way we can configure the
> flow
> > > control that blocks only one producer? Do you have any recommendation
> on
> > > what changes I should make to the config?
> > >
> > > On Wed, Jan 2, 2013 at 12:33 PM, Christian Posta
> > > <christian.posta@gmail.com>wrote:
> > >
> > > > Enqueue count = number of messages sent to the queue
> > > > Dequeue count = number of messages ack'd by the consumer
> > > > Inflight count = number of messages sent to consumer's session (and
> are
> > > not
> > > > ack'd by consumer)
> > > > Dispatch count = messages sent to the consumer (dequeue + inflight)
> > > >
> > > > I asked about the async sends because the producer flow control you
> hit
> > > was
> > > > the kind that will block the entire connection (instead of just
> > blocking
> > > > that one producer).
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Jan 2, 2013 at 12:41 PM, Mohit Anchlia <
> mohitanchlia@gmail.com
> > > > >wrote:
> > > >
> > > > > Thanks again! Is the dispatch queue size the one that relates to
> > what's
> > > > > held in memory?
> > > > >
> > > > > We are using mule and the endpoint has no transactions configured.
> > What
> > > > did
> > > > > you mean by forcing async?
> > > > >
> > > > > On Wed, Jan 2, 2013 at 11:06 AM, Christian Posta
> > > > > <christian.posta@gmail.com>wrote:
> > > > >
> > > > > > Correct. The memoryUsage is increased when messages are held
in
> > > memory.
> > > > > > This happens when messages are paged in from the store. They
are
> > then
> > > > > > dispatched to consumers and held until the consumer acks.
> > > > > >
> > > > > > In your case looks like a particular destination (eventsEndpoint)
> > is
> > > > > > filling up somehow. Can you look in JMX and see how many
> enqueued,
> > > > > > dispatched, dequeued, inflight etc? Also check to see how many
> > > > > > subscriptions to that destination, and see what those
> > > dispatch-counter
> > > > > and
> > > > > > dispatch queue size looks like.
> > > > > >
> > > > > > BTW any chance you're sending (producers) using a transaction,
or
> > > > forcing
> > > > > > sends to be async?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Jan 2, 2013 at 11:51 AM, Mohit Anchlia <
> > > mohitanchlia@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > So if I understand correctly I shouldn't be seeing this
> behaviour
> > > > > unless
> > > > > > > something is being held in memory. We ack immediately so
I am
> > kind
> > > of
> > > > > > > wondering what might be going on here. Is there a way to
find
> out
> > > > what
> > > > > > > might be happening here?
> > > > > > >
> > > > > > > On Wed, Jan 2, 2013 at 10:03 AM, Christian Posta
> > > > > > > <christian.posta@gmail.com>wrote:
> > > > > > >
> > > > > > > > What info can you provide about how clients are consuming
> from
> > > the
> > > > > > > queues?
> > > > > > > > Any chance there are a bunch of messages "inflight"
which
> > haven't
> > > > > been
> > > > > > > > ack'd by the consumer?
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jan 2, 2013 at 10:20 AM, Mohit Anchlia <
> > > > > mohitanchlia@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > I see these messages being logged. Queues are
setup to use
> > > > > persistent
> > > > > > > > > store. I am wondering what's causing this messages
to get
> > > > blocked.
> > > > > I
> > > > > > > see
> > > > > > > > > there is a memoryLimit but if I am using persistent=true
> then
> > > > > should
> > > > > > > this
> > > > > > > > > be the desired behaviour?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 2013-01-02 05:06:13,388 | INFO |
> > > > > > > > > Usage(default:memory:queue://eventsEndpoint:memory)
> > > > > percentUsage=0%,
> > > > > > > > > usage=0, limit=20971520,
> > > > > > > > > percentUsageMinDelta=1%;Parent:Usage(default:memory)
> > > > > > percentUsage=106%,
> > > > > > > > > usage=22366029, limit=20971520, percentUsageMinDelta=1%:
> > Usage
> > > > > > Manager
> > > > > > > > > Memory Limit reached. Producer
> > > > > > > > > (ID:pfdamq301.ie.net-54582-1351809386355-2:1:1:1)
stopped
> to
> > > > > prevent
> > > > > > > > > flooding queue://eventsEndpoint. See
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >    <broker xmlns="http://activemq.apache.org/schema/core"
> > > > > > > > > brokerName="static-broker1" persistent="true"
> > > > > > > > > dataDirectory="${activemq.data}">
> > > > > > > > >
> > > > > > > > >         <!-- Destination specific policies
using
> destination
> > > > names
> > > > > or
> > > > > > > > > wildcards -->
> > > > > > > > >         <destinationPolicy>
> > > > > > > > >             <policyMap>
> > > > > > > > >                 <policyEntries>
> > > > > > > > >                     <policyEntry queue=">"
> > > > > producerFlowControl="true"
> > > > > > > > > memoryLimit="20mb">
> > > > > > > > >                         <deadLetterStrategy>
> > > > > > > > >                           <individualDeadLetterStrategy
> > > > > > > > queuePrefix="DLQ."
> > > > > > > > > useQueueForQueueMessages="true" />
> > > > > > > > >                         </deadLetterStrategy>
> > > > > > > > >                     </policyEntry>
> > > > > > > > >                     <policyEntry topic=">"
> > > > > producerFlowControl="true"
> > > > > > > > > memoryLimit="20mb">
> > > > > > > > >                     </policyEntry>
> > > > > > > > >                 </policyEntries>
> > > > > > > > >             </policyMap>
> > > > > > > > >         </destinationPolicy>
> > > > > > > > >
> > > > > > > > >         <!-- Use the following to configure
how ActiveMQ is
> > > > exposed
> > > > > > in
> > > > > > > > JMX
> > > > > > > > > -->
> > > > > > > > >         <managementContext>
> > > > > > > > >             <managementContext createConnector="true"/>
> > > > > > > > >         </managementContext>
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > *Christian Posta*
> > > > > > > > http://www.christianposta.com/blog
> > > > > > > > twitter: @christianposta
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Christian Posta*
> > > > > > http://www.christianposta.com/blog
> > > > > > twitter: @christianposta
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Christian Posta*
> > > > http://www.christianposta.com/blog
> > > > twitter: @christianposta
> > > >
> > >
> >
> >
> >
> > --
> > *Christian Posta*
> > http://www.christianposta.com/blog
> > twitter: @christianposta
> >
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message