activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Re: Memory limit errors
Date Thu, 03 Jan 2013 22:43:59 GMT
Just one more question, is there a way to force acknowledgment of a message
or send it to DLQ? I am wondering what safeguards we can put in place in
active mq so that if and when this occurs it doesn't impact all other
clients.

On Thu, Jan 3, 2013 at 7:57 AM, Mohit Anchlia <mohitanchlia@gmail.com>wrote:

> Yes it's really helpful. Thanks again
>
>
> On Thu, Jan 3, 2013 at 5:33 AM, Christian Posta <christian.posta@gmail.com
> > wrote:
>
>> Good call Claus. I've updated the wiki page here:
>>
>> http://activemq.apache.org/how-do-i-find-the-size-of-a-queue.html
>>
>> Should be a couple hours to propagate through.
>>
>> Thanks!
>>
>>
>> On Thu, Jan 3, 2013 at 1:05 AM, Claus Ibsen <claus.ibsen@gmail.com>
>> wrote:
>>
>> > On Wed, Jan 2, 2013 at 9:33 PM, Christian Posta
>> > <christian.posta@gmail.com> wrote:
>> > > Enqueue count = number of messages sent to the queue
>> > > Dequeue count = number of messages ack'd by the consumer
>> > > Inflight count = number of messages sent to consumer's session (and
>> are
>> > not
>> > > ack'd by consumer)
>> > > Dispatch count = messages sent to the consumer (dequeue + inflight)
>> > >
>> >
>> > Nice summary. Christian I think you should add that as a FAQ
>> > http://activemq.apache.org/faq
>> >
>> >
>> > > I asked about the async sends because the producer flow control you
>> hit
>> > was
>> > > the kind that will block the entire connection (instead of just
>> blocking
>> > > that one producer).
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Wed, Jan 2, 2013 at 12:41 PM, Mohit Anchlia <
>> mohitanchlia@gmail.com
>> > >wrote:
>> > >
>> > >> Thanks again! Is the dispatch queue size the one that relates to
>> what's
>> > >> held in memory?
>> > >>
>> > >> We are using mule and the endpoint has no transactions configured.
>> What
>> > did
>> > >> you mean by forcing async?
>> > >>
>> > >> On Wed, Jan 2, 2013 at 11:06 AM, Christian Posta
>> > >> <christian.posta@gmail.com>wrote:
>> > >>
>> > >> > Correct. The memoryUsage is increased when messages are held in
>> > memory.
>> > >> > This happens when messages are paged in from the store. They are
>> then
>> > >> > dispatched to consumers and held until the consumer acks.
>> > >> >
>> > >> > In your case looks like a particular destination (eventsEndpoint)
>> is
>> > >> > filling up somehow. Can you look in JMX and see how many enqueued,
>> > >> > dispatched, dequeued, inflight etc? Also check to see how many
>> > >> > subscriptions to that destination, and see what those
>> dispatch-counter
>> > >> and
>> > >> > dispatch queue size looks like.
>> > >> >
>> > >> > BTW any chance you're sending (producers) using a transaction,
or
>> > forcing
>> > >> > sends to be async?
>> > >> >
>> > >> >
>> > >> >
>> > >> > On Wed, Jan 2, 2013 at 11:51 AM, Mohit Anchlia <
>> > mohitanchlia@gmail.com
>> > >> > >wrote:
>> > >> >
>> > >> > > So if I understand correctly I shouldn't be seeing this behaviour
>> > >> unless
>> > >> > > something is being held in memory. We ack immediately so
I am
>> kind
>> > of
>> > >> > > wondering what might be going on here. Is there a way to
find out
>> > what
>> > >> > > might be happening here?
>> > >> > >
>> > >> > > On Wed, Jan 2, 2013 at 10:03 AM, Christian Posta
>> > >> > > <christian.posta@gmail.com>wrote:
>> > >> > >
>> > >> > > > What info can you provide about how clients are consuming
from
>> the
>> > >> > > queues?
>> > >> > > > Any chance there are a bunch of messages "inflight"
which
>> haven't
>> > >> been
>> > >> > > > ack'd by the consumer?
>> > >> > > >
>> > >> > > >
>> > >> > > > On Wed, Jan 2, 2013 at 10:20 AM, Mohit Anchlia <
>> > >> mohitanchlia@gmail.com
>> > >> > > > >wrote:
>> > >> > > >
>> > >> > > > > I see these messages being logged. Queues are setup
to use
>> > >> persistent
>> > >> > > > > store. I am wondering what's causing this messages
to get
>> > blocked.
>> > >> I
>> > >> > > see
>> > >> > > > > there is a memoryLimit but if I am using persistent=true
then
>> > >> should
>> > >> > > this
>> > >> > > > > be the desired behaviour?
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > 2013-01-02 05:06:13,388 | INFO |
>> > >> > > > > Usage(default:memory:queue://eventsEndpoint:memory)
>> > >> percentUsage=0%,
>> > >> > > > > usage=0, limit=20971520,
>> > >> > > > > percentUsageMinDelta=1%;Parent:Usage(default:memory)
>> > >> > percentUsage=106%,
>> > >> > > > > usage=22366029, limit=20971520, percentUsageMinDelta=1%:
>> Usage
>> > >> > Manager
>> > >> > > > > Memory Limit reached. Producer
>> > >> > > > > (ID:pfdamq301.ie.net-54582-1351809386355-2:1:1:1)
stopped to
>> > >> prevent
>> > >> > > > > flooding queue://eventsEndpoint. See
>> > >> > > > >
>> > >> > > > >
>> > >> > > > >    <broker xmlns="http://activemq.apache.org/schema/core"
>> > >> > > > > brokerName="static-broker1" persistent="true"
>> > >> > > > > dataDirectory="${activemq.data}">
>> > >> > > > >
>> > >> > > > >         <!-- Destination specific policies using
destination
>> > names
>> > >> or
>> > >> > > > > wildcards -->
>> > >> > > > >         <destinationPolicy>
>> > >> > > > >             <policyMap>
>> > >> > > > >                 <policyEntries>
>> > >> > > > >                     <policyEntry queue=">"
>> > >> producerFlowControl="true"
>> > >> > > > > memoryLimit="20mb">
>> > >> > > > >                         <deadLetterStrategy>
>> > >> > > > >                           <individualDeadLetterStrategy
>> > >> > > > queuePrefix="DLQ."
>> > >> > > > > useQueueForQueueMessages="true" />
>> > >> > > > >                         </deadLetterStrategy>
>> > >> > > > >                     </policyEntry>
>> > >> > > > >                     <policyEntry topic=">"
>> > >> producerFlowControl="true"
>> > >> > > > > memoryLimit="20mb">
>> > >> > > > >                     </policyEntry>
>> > >> > > > >                 </policyEntries>
>> > >> > > > >             </policyMap>
>> > >> > > > >         </destinationPolicy>
>> > >> > > > >
>> > >> > > > >         <!-- Use the following to configure
how ActiveMQ is
>> > exposed
>> > >> > in
>> > >> > > > JMX
>> > >> > > > > -->
>> > >> > > > >         <managementContext>
>> > >> > > > >             <managementContext createConnector="true"/>
>> > >> > > > >         </managementContext>
>> > >> > > > >
>> > >> > > >
>> > >> > > >
>> > >> > > >
>> > >> > > > --
>> > >> > > > *Christian Posta*
>> > >> > > > http://www.christianposta.com/blog
>> > >> > > > twitter: @christianposta
>> > >> > > >
>> > >> > >
>> > >> >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > *Christian Posta*
>> > >> > http://www.christianposta.com/blog
>> > >> > twitter: @christianposta
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > *Christian Posta*
>> > > http://www.christianposta.com/blog
>> > > twitter: @christianposta
>> >
>> >
>> >
>> > --
>> > Claus Ibsen
>> > -----------------
>> > Red Hat, Inc.
>> > FuseSource is now part of Red Hat
>> > Email: cibsen@redhat.com
>> > Web: http://fusesource.com
>> > Twitter: davsclaus
>> > Blog: http://davsclaus.com
>> > Author of Camel in Action: http://www.manning.com/ibsen
>> >
>>
>>
>>
>> --
>> *Christian Posta*
>> http://www.christianposta.com/blog
>> twitter: @christianposta
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message