activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: why is activeMQ stalled?
Date Fri, 15 Jan 2010 10:25:56 GMT
A thread dump of the broker jvm, when the clients are stalled, would help
diagnose what is going on. btw, what activemq version are you using?

2010/1/15 David Delbecq <delbd@oma.be>

> Le 15/01/10 10:24, Dejan Bosanac a écrit :
> > Hi David,
> >
> > you're probably blocked by "producer flow control". Take a look here
> > for more info: http://activemq.apache.org/producer-flow-control.html
> >
> > Cheers
>
> I already took a look at it, read it three time to be sure i understand.
> It's quite technical but beyond that, i don't think this apply for me.
> My queues are all empty!!! (they stalled when there were 50 messages
> pending but where finally emptied), connections are not shared by
> several queues and am not sure how to implement the cursor part of
> documentation. There is no "slow consumer" that could influence it at
> that right moment (customers are running an waiting for new messages as
> you can see in capture). Moreover my messages are of about 1k payload
> (about 10 small text entries in a MapMessage), and my activeMQ config is
> like this
>                    <policyEntry queue=">" memoryLimit="500mb"/>
>                    <policyEntry topic=">" memoryLimit="500mb">
> ....
>        <systemUsage>
>            <systemUsage>
>                <memoryUsage>
>                    <memoryUsage limit="20 mb"/>
>                </memoryUsage>
>                <storeUsage>
>                    <storeUsage limit="1 gb" name="foo"/>
>                </storeUsage>
>                <tempUsage>
>                    <tempUsage limit="100 mb"/>
>                </tempUsage>
>            </systemUsage>
>        </systemUsage>
>
>
> There is no way a bunch of 10 pending messages could fill a 1gb store,
> isn't it? What config part of activemq do i need to change to ensure
> that it can store at least about 10000 persistent messages (that makes
> only 10Mb of datas!!) before failing? Currently it fails and  is never
> recovering without a restart of broker when there are only 50 small
> pending messages in the queue. As a workaround i have set
> "sendFailIfNoSpace" to true, so my client will fail instead of stalling,
> i'll see when this happen more quickly like that....
>
>
> Here is broker part of my config file, can someone help me fix this?
> Thank you:
>
>
>    <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.base}/data"
> persistent="true">
>        <plugins>
>        <simpleAuthenticationPlugin>
>          <users>
>                <authenticationUser username="admin" password="******"
>                        groups="users,admins"/>
>                <authenticationUser username="sharkTool" password="********"
>                        groups="serverTools"/>
>                <authenticationUser username="adminConsole"
> password="*********"
>                        groups="serverTools"/>
>          </users>
>        </simpleAuthenticationPlugin>
>      <authorizationPlugin>
>        <map>
>          <authorizationMap>
>            <authorizationEntries>
>              <authorizationEntry queue=">" read="admins" write="admins"
> admin="admins" />
>              <authorizationEntry queue="WF.>" read="users,serverTools"
> write="serverTools" admin="admins,serverTools" />
>
>              <authorizationEntry topic=">" read="admins" write="admins"
> admin="admins" />
>              <authorizationEntry topic="WF.>" read="users,serverTools"
> write="serverTools" admin="admins,serverTools" />
>
>              <authorizationEntry topic="ActiveMQ.Advisory.>"
> read="guests,users,serverTools" write="guests,users,serverTools"
> admin="guests,users,serverTools"/>
>            </authorizationEntries>
>
>            <!-- let's assign roles to temporary destinations. comment
> this entry if we don't want any roles assigned to temp destinations
>            <tempDestinationAuthorizationEntry>
>              <tempDestinationAuthorizationEntry
> read="tempDestinationAdmins" write="tempDestinationAdmins"
> admin="tempDestinationAdmins"/>
>           </tempDestinationAuthorizationEntry>          -->
>          </authorizationMap>
>        </map>
>      </authorizationPlugin>
>
>
>        </plugins>
>        <!-- Destination specific policies using destination names or
> wildcards -->
>        <destinationPolicy>
>            <policyMap>
>                <policyEntries>
>                    <policyEntry queue=">" memoryLimit="500mb"/>
>                    <policyEntry topic=">" memoryLimit="500mb">
>                      <!-- you can add other policies too such as these
>                        <dispatchPolicy>
>                            <strictOrderDispatchPolicy/>
>                        </dispatchPolicy>
>                        <subscriptionRecoveryPolicy>
>                            <lastImageSubscriptionRecoveryPolicy/>
>                        </subscriptionRecoveryPolicy>
>                      -->
>                    </policyEntry>
>                </policyEntries>
>            </policyMap>
>        </destinationPolicy>
>
>        <!-- Use the following to configure how ActiveMQ is exposed in
> JMX -->
>        <managementContext>
>            <managementContext createConnector="false"/>
>        </managementContext>
>
>        <!-- The store and forward broker networks ActiveMQ will listen
> to -->
>        <networkConnectors>
>            <!-- by default just auto discover the other brokers -->
>            <!-- <networkConnector name="default-nc"
> uri="multicast://default"/>-->
>            <!-- Example of a static configuration:
>            <networkConnector name="host1 and host2"
> uri="static://(tcp://host1:61616,tcp://host2:61616)"/>
>            -->
>        </networkConnectors>
>
>        <persistenceAdapter>
>            <amqPersistenceAdapter syncOnWrite="false"
> directory="${activemq.base}/data" maxFileLength="20 mb"/>
>        </persistenceAdapter>
>
>
>
>        <sslContext>
>            <sslContext keyStore="file:${activemq.base}/conf/broker.ks"
> keyStorePassword="password"
> trustStore="file:${activemq.base}/conf/broker.ts"
> trustStorePassword="password"/>
>        </sslContext>
>
>        <!--  The maximum about of space the broker will use before
> slowing down producers -->
>        <systemUsage>
>            <systemUsage>
>                <memoryUsage>
>                    <memoryUsage limit="20 mb"/>
>                </memoryUsage>
>                <storeUsage>
>                    <storeUsage limit="1 gb" name="foo"/>
>                </storeUsage>
>                <tempUsage>
>                    <tempUsage limit="100 mb"/>
>                </tempUsage>
>            </systemUsage>
>        </systemUsage>
>
>
>        <!-- The transport connectors ActiveMQ will listen to -->
>        <transportConnectors>
>            <transportConnector name="openwire"
> uri="tcp://localhost:61616"/>
>            <!-- <transportConnector name="ssl"
> uri="ssl://localhost:61617"/>
>            <transportConnector name="stomp" uri="stomp://localhost:61613"/>
>            <transportConnector name="xmpp"
> uri="xmpp://localhost:61222"/> -->
>        </transportConnectors>
>        <destinations>
>            <queue physicalName="WF.Regraph" />
>              <queue physicalName="WF.Notify"/>
>              <topic physicalName="WF.Events"/>
>        </destinations>
>            <destinationInterceptors>
>      <virtualDestinationInterceptor>
>        <virtualDestinations>
>          <compositeQueue name="WF.IncomingEvents">
>            <forwardTo>
>              <queue physicalName="WF.Regraph"/>
>              <queue physicalName="WF.Notify"/>
>              <topic physicalName="WF.Events" />
>            </forwardTo>
>          </compositeQueue>
>        </virtualDestinations>
>      </virtualDestinationInterceptor>
>    </destinationInterceptors>
>
>
>    </broker>
>
>
> > --
> >
> >
> > On Fri, Jan 15, 2010 at 10:24 AM, David Delbecq <delbd@oma.be
> > <mailto:delbd@oma.be>> wrote:
> >
> >     Hello,
> >
> >     can someone explain me why my activeMQ client is stalled in such
> >     situation (i have 10 threads blocked like this having as
> >     consequence a complete blocking of my client application):
> >
> >     "http-8081-Processor6" daemon prio=10 tid=00841c38 nid=512
> >     lwp_id=2426611 waiting on condition [236f3000..236f153c]
> >        java.lang.Thread.State: WAITING (parking)
> >             at sun.misc.Unsafe.park(Native Method)
> >             - parking to wait for  <40ae1828> (a
> >
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> >             at
> >     java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> >             at
> >
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1889)
> >             at
> >
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
> >             at
> >
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
> >             at
> >
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
> >             at
> >
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
> >             at
> >     org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1659)
> >             - locked <40ae01e8> (a java.lang.Object)
> >             at
> >
> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227)
> >             at
> >
> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
> >             at
> >     be.rmi.shark.JMSEventManager.sendEvent(JMSEventManager.java:146)
> >
> >
> >     considering that the activeMQ lists size, as referenced by
> >     activeMQ admin interface are as follow:
> >     Name      Number Of Pending Messages      Number Of Consumers
> >     Messages Sent     Messages Received       Views           Operations
> >     WF.Regraph        0       1       1714    1720    Browse  Send To
> Purge Delete
> >     WF.Notify         0       1       1696    1742    Browse  Send To
> Purge Delete
> >     WF.IncomingEvents         0       0       0       0       Browse
>  Send To Purge Delete
> >
> >
> >     Is this to be considered normal that i must restart activeMQ
> >     producers on a regular basis because of those this stalling? Where
> >     can i find clear documentation on proper configuration of activeMQ
> >     to prevent this stalling? Thank you.
> >
> >     --
> >     David Delbecq
> >     ICT
> >     Institut Royal Météorologique
> >     Ext:557
> >
> >
> >
>
>
> --
> David Delbecq
> ICT
> Institut Royal Météorologique
> Ext:557
>
>


-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message