activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Anderson <manderso...@gmail.com>
Subject Re: Memory and Temp Usage Questions
Date Tue, 27 Nov 2012 14:08:06 GMT
The prefetch size was set on the network connector as we were getting
messages about slow consumers across the network bridge.

As far as I can see the network bridge had not failed. The connector
entries in the log are for a client subscription that will also have the
topic prefetch set to 32766. I am trying to get logs from the client.

The broker on the other end of the bridge uses the same configuration.


On 27 November 2012 13:41, Christian Posta <christian.posta@gmail.com>wrote:

> Answers to your questions:
>
> 1) Not sure yet
> 2) Because at the moment, send fail if no space is only triggered when
> producer flow control is on (at least for this case, topics)
> 3) like gtully said, connections could not be shutdown if they are blocked
> somehow
>
> I noticed in your config you explicitly set the prefetch on the network
> connector to 32766. The default for network connectors is 1000 and the
> default for regular topics is Short.MAX_VALUE (which is 32767). Since the
> bridge doesn't have a prefetch buffer like normal clients do, setting the
> prefetch to 32766 could end up flooding it. Any reason why you have it set
> to 32766?
>
> So TopicSubscriptions should always have the broker's main memory usage. If
> it has the destination's memory limit, then something went wrong. Like Gary
> said, the pending message cursor's messages would be spooled to disk when
> the main memory limit reaches its high water mark (70% by default).... but
> that appears to not have happened in this case.
>
> Are there any indications that the TopicSubscription is for the network
> bridge? Or maybe that the network bridge failed somehow? I see that the
> dispatched count is the same as what you've set for you prefertch on the
> bridge, but if anything else can point to that it might be helpful. For
> example, are those port numbers on the transport connector logs for the
> network bridge?
>
> How is the broker on the other end of the bridge configured? Same?
>
>
> On Fri, Nov 23, 2012 at 8:36 AM, Mark Anderson <manderson23@gmail.com
> >wrote:
>
> > I have ActiveMQ 5.6.0 configured as follows:
> >
> > Producer Flow Control = false
> > Send Fail If No Space = true
> > Memory Usage Limit = 128Mb
> > Temp Usage Limit = 1Gb
> >
> > All my messages are non-persistent. The temp usage is configured to
> handle
> > spikes/slow consumers when processing messages.
> >
> > I continually see the following in the logs:
> >
> > WARN  Nov 20 20:55:47 (13748874 [InactivityMonitor Async Task:
> > java.util.concurrent.ThreadPoolExecutor$Worker@7ea0e15b[State = 0, empty
> > queue]] org.apac
> > he.activemq.broker.TransportConnection.Transport) Transport Connection
> to:
> > tcp://192.168.2.103:35186 failed: java.net.SocketException: Broken pipe
> > INFO  Nov 20 20:55:51 (13752162 [ActiveMQ Transport: tcp:///
> > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The
> > connection to 'tcp:
> > //192.168.2.103:35166' is taking a long time to shutdown.
> > INFO  Nov 20 20:55:56 (13757162 [ActiveMQ Transport: tcp:///
> > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The
> > connection to 'tcp:
> > //192.168.2.103:35166' is taking a long time to shutdown.
> > INFO  Nov 20 20:56:01 (13762162 [ActiveMQ Transport: tcp:///
> > 192.168.2.103:35168] org.apache.activemq.broker.TransportConnection) The
> > connection to 'tcp:
> > //192.168.2.103:35166' is taking a long time to shutdown.
> >
> > I'm not sure why the connection will never shutdown.
> >
> > I then see the following message:
> >
> > org.apache.activemq.broker.region.TopicSubscription) TopicSubscription:
> > consumer=ID:linux-5ks2-57958-1353426643811-3:1:378:1, destinations=1,
> > dispatched=32766, delivered=0, matched=0, discarded=0: Pending message
> > cursor
> >
> >
> [org.apache.activemq.broker.region.cursors.FilePendingMessageCursor@4c41cfa2
> > ]
> > is full, temp usage (0%) or memory usage (211%) limit reached, blocking
> > message add() pending the release of resources.
> >
> > This leads me to the following questions:
> >
> > 1) Why would the memory usage be 211% while temp usage is 0%.
> > 2) The thread dump shows that send calls on producers are blocking. Why
> > would they not throw exceptions when send fail if no space = true?
> > 3) Would the issue with connection shutdown contribute to the memory
> usage?
> >
> > Thanks,
> > Mark
> >
>
>
>
> --
> *Christian Posta*
> http://www.christianposta.com/blog
> twitter: @christianposta
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message