Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: pass (athena.apache.org: domain of christian.posta@gmail.com
 designates 209.85.223.171 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <50BE64A9.2040604@d3s.mff.cuni.cz>
References: <50BE64A9.2040604@d3s.mff.cuni.cz>
Date: Wed, 5 Dec 2012 06:35:54 -0700
Message-ID: 
 <CAKS7T5MP1hPzUBoh4buKrWM_N6gftp15Zwo5JrtF3kRas41uNw@mail.gmail.com>
Subject: Re: Activemq 5.4.2 hangs when the temp disk usage is used
From: Christian Posta <christian.posta@gmail.com>
To: "users@activemq.apache.org" <users@activemq.apache.org>
Content-Type: multipart/alternative; boundary=20cf303f64eac732c804d01b0fd7

--20cf303f64eac732c804d01b0fd7
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Interesting. Your scenario sounds like PFC is blocking the connection. Not
sure yet why that would happen. Your config looks okay. Do you still have
the thread dumps?

Are the producer/consumer on the same connection? If you have a test case
you can post, please do. Would make it easy to figure what the problem is.


On Tue, Dec 4, 2012 at 2:01 PM, Tom=C3=A1=C5=A1 Martinec <
tomas.martinec@d3s.mff.cuni.cz> wrote:

> Hello,
>
> I am trying to setup activemq 5.4.2 and I experience hangs when the broke=
r
> fills its heap and starts using storage. In my scenario I have one messag=
e
> producer and one message consumer connected. The producer is blocked and
> the consumer does not receive any messages. I have been able to run the
> broker under debug mode and identify what the relevant threads are doing:
>
> The task for iteration of the queue waits for the messagesLock on the lin=
e
> 1404 in http://svn.apache.org/viewvc/**activemq/tags/activemq-5.4.2/**
> activemq-core/src/main/java/**org/apache/activemq/broker/**
> region/Queue.java?view=3Dmarkup<http://svn.apache.org/viewvc/activemq/tag=
s/activemq-5.4.2/activemq-core/src/main/java/org/apache/activemq/broker/reg=
ion/Queue.java?view=3Dmarkup>.
>
>
> The thread that stores messages into the queue waits on the line 219 of
> http://svn.apache.org/viewvc/**activemq/tags/activemq-5.4.2/**
> activemq-core/src/main/java/**org/apache/activemq/broker/**region/cursors=
/
> **FilePendingMessageCursor.java?**view=3Dmarkup<http://svn.apache.org/vie=
wvc/activemq/tags/activemq-5.4.2/activemq-core/src/main/java/org/apache/act=
ivemq/broker/region/cursors/FilePendingMessageCursor.java?view=3Dmarkup>for=
 free space. Note that the thread has the messagesLock locked and the
> temporaly usage is always 103%, and therefore, it never returns.
>
> Is it possible that I encountered something like a deadlock? Perhaps, I
> messed up something in the configuration.
>
> I will be thankful for advice.
> Tomas
>
> Here is the configuration of the broker:
>
> <beans
>   xmlns=3D"http://www.**springframework.org/schema/**beans<http://www.spr=
ingframework.org/schema/beans>
> "
>   xmlns:amq=3D"http://activemq.**apache.org/schema/core<http://activemq.a=
pache.org/schema/core>
> "
>   xmlns:xsi=3D"http://www.w3.org/**2001/XMLSchema-instance<http://www.w3.=
org/2001/XMLSchema-instance>
> "
>   xsi:schemaLocation=3D"http://**www.springframework.org/**schema/beans<h=
ttp://www.springframework.org/schema/beans>
> http://www.springframework.**org/schema/beans/spring-beans-**2.0.xsd<http=
://www.springframework.org/schema/beans/spring-beans-2.0.xsd>
>   http://activemq.apache.org/**schema/core<http://activemq.apache.org/sch=
ema/core>
> http://activemq.apache.org/**schema/core/activemq-core.xsd<http://activem=
q.apache.org/schema/core/activemq-core.xsd>
> "**>
>
>     <!-- Allows us to use system properties as variables in this
> configuration file -->
>     <bean class=3D"org.springframework.**beans.factory.config.**PropertyP=
laceholderConfigurer"
> />
>
>     <!--
>         The <broker> element is used to configure the ActiveMQ broker.
>
>         advisories incurr a hit with every add connection|destination|**
> producer|consumer
>           - client side:  providerUrl =3D tcp://localhost:61616?jms.**
> watchTopicAdvisories=3Dfalse
>         statistics have a small impact on concurrency so disable
>         schedualer has a small impact as it checks for the presence three
> properties on each message
>         jmx adds a bunch of wrapper classes
>
>         ACTIVEMQ_OPTS_MEMORY=3D"-Xmx11g -Xms6g -Dorg.apache.activemq.**Us=
eDedicatedTaskRunner=3Dfalse
> -Djava.util.logging.config.**file=3Dlogging.properties -XX:+UseLargePages=
"
>     -->
>     <broker xmlns=3D"http://activemq.apache.**org/schema/core<http://acti=
vemq.apache.org/schema/core>"
> brokerName=3D"localhost"
>         dataDirectory=3D"/dasd/martinec/**kahadb-storage/activemq-data"
>         destroyApplicationContextOnSto**p=3D"true"
>         deleteAllMessagesOnStartup=3D"**true"
>         advisorySupport=3D"false"
>         enableStatistics=3D"false"
>         schedulerSupport=3D"false"
>         useJmx=3D"true"
>         persistent=3D"true">
>
>         <destinationPolicy>
>             <policyMap>
>               <policyEntries>
>                 <!--
>                     producerFlowControll thread unnecessary with no memor=
y
> limits
>                     with no failover, no need to suppress duplicates so
> audit can be disabled
>                     message expiry thread unnecessary as normal dispatch
> will handle expiry
>                 -->
>                 <policyEntry topic=3D">" producerFlowControl=3D"false"
> enableAudit=3D"false" expireMessagesPeriod=3D"0"></**policyEntry>
>                 <policyEntry queue=3D">" producerFlowControl=3D"true"
> memoryLimit=3D"50mb" enableAudit=3D"false" expireMessagesPeriod=3D"0">
>
>                 </policyEntry>
>               </policyEntries>
>             </policyMap>
>         </destinationPolicy>
>
>         <!--
>             The managementContext is used to configure how ActiveMQ is
> exposed in
>             JMX. By default, ActiveMQ uses the MBean server that is
> started by
>             the JVM. For more information, see:
>
>             http://activemq.apache.org/**jmx.html<http://activemq.apache.=
org/jmx.html>
>         -->
>         <managementContext>
>             <managementContext createConnector=3D"true" connectorPort=3D"=
1399"
> rmiServerPort=3D"1399"/>
>         </managementContext>
>
>         <persistenceAdapter>
>             <kahaPersistenceAdapter directory=3D"/dasd/martinec/**
> kahadb-storage"/>
>         </persistenceAdapter>
>
>         <systemUsage>
>             <systemUsage>
>                 <memoryUsage>
>                     <!-- the only queue has 50 mb, so this should be
> enough -->
>                     <memoryUsage limit=3D"200 mb"/>
>                 </memoryUsage>
>                 <storeUsage>
>                     <!-- no special reason for this -->
>                     <storeUsage limit=3D"1 gb"/>
>                 </storeUsage>
>                 <tempUsage>
>                     <!-- this MUST be above the journal file size; see
>          http://tmielke.blogspot.cz/**2011/02/observations-on-**
> activemqs-temp-storage.html<http://tmielke.blogspot.cz/2011/02/observatio=
ns-on-activemqs-temp-storage.html>
>                      -->
>                     <tempUsage limit=3D"100 mb"/>
>                 </tempUsage>
>             </systemUsage>
>         </systemUsage>
>
>         <!--
>             The transport connectors expose ActiveMQ over a given protoco=
l
> to
>             clients and other brokers. For more information, see:
>
>             http://activemq.apache.org/**configuring-transports.html<http=
://activemq.apache.org/configuring-transports.html>
>         -->
>         <transportConnectors>
>             <transportConnector name=3D"openwire" uri=3D"tcp://0.0.0.0:61=
616
> "/>
>         </transportConnectors>
>     </broker>
>
>     <import resource=3D"jetty.xml"/>
>
> </beans>
>


--=20
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

--20cf303f64eac732c804d01b0fd7--