activemq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-6240) Producer cannot be terminated when slow consumer is detected
Date Mon, 11 Apr 2016 11:56:25 GMT

    [ https://issues.apache.org/jira/browse/AMQ-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234939#comment-15234939
] 

Gary Tully commented on AMQ-6240:
---------------------------------

{code}"ActiveMQ Transport: tcp:///127.0.0.1:65190@61616" #52 daemon prio=5 os_prio=0 tid=0x000000005ac6b800
nid=0x20fc waiting on condition [0x000000006035e000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000c050b598> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
	at org.apache.activemq.usage.Usage.waitForSpace(Usage.java:103)
	at org.apache.activemq.usage.StoreUsage.waitForSpace(StoreUsage.java:92)
	at org.apache.activemq.usage.StoreUsage.waitForSpace(StoreUsage.java:87)
	at org.apache.activemq.broker.region.BaseDestination.waitForSpace(BaseDestination.java:686)
	at org.apache.activemq.broker.region.Queue.checkUsage(Queue.java:899)
	at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:831)
	at org.apache.activemq.broker.region.Queue.send(Queue.java:731)
	at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:503)
	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:468)
	at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:293)
	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:153)
	at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
	at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:293)
	at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:158)
	at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:546)
	at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:768)
	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:338)
	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:125)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:300)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:233)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
	at java.lang.Thread.run(Thread.java:745){code}


the transport thread is busy waiting for space, blocked, so the next command, the rollback/close
cannot be read from the socket.
Without broker side async blocking, via producerFlowControl or the error on no space, the
default behaviour is to block the transport thread.

Because the broker config cannot be changed I think there are two client options:
1) use async send and a small producerWindow, this will cause your client to block pending
space on the broker for pending messages. However having peeked at the org.apache.activemq.ActiveMQMessageProducer#send(javax.jms.Destination,
javax.jms.Message, int, int, long, org.apache.activemq.AsyncCallback) - there is no timeout
limit, so an interrupted exception is all that will stop that wait. not so good.

2) force a close of the connection via the underlying transport, so that the  jms connection
sees a socket closure and will report an error. This should avoid the normal closure processing
after you see a send timeout.
That is a bit of a hack, using an activemq narrow api. There are some examples of how to do
it in the unit tests:
https://github.com/apache/activemq/blob/e2b4ca2c59a316be91cf391c758a10e518dd3a1f/activemq-unit-tests/src/test/java/org/apache/activemq/transport/failover/FailoverTimeoutTest.java#L210


> Producer cannot be terminated when slow consumer is detected
> ------------------------------------------------------------
>
>                 Key: AMQ-6240
>                 URL: https://issues.apache.org/jira/browse/AMQ-6240
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Connector
>    Affects Versions: 5.13.2
>         Environment: The test was run using an out of the box install of ActiveMQ 5.13.2
on Windows 7 Professional (64 bit). 
> The java version used was jdk1.8.0_60
>            Reporter: Ken Hall
>         Attachments: brokertd.zip
>
>
> There is a discussion relating to the bug here - http://activemq.2283324.n4.nabble.com/Force-Producer-to-fail-when-blocked-by-slow-consumer-td4710264.html.
Test code to reproduce the problem  has also been uploaded to this discussion.
> We have  a situation where we have a producer which is feeding a slow consumer.
> The consumer belongs to a third party so we do not have control over the broker configuration.
> We would like to terminate our producer but we find that we cannot do so. We can set
the timeout on the connection and we then trap the timeout exception. The problem is that
when we try to close the session it attempts to rollback but cannot do so as the rollback
blocks.
> We have created a small test that is included in the zip (mentioned above) of an Eclipse
project.The zip includes three classes. Call the main method of RunTest to run the test. The
activemq.xml used is in the root directory of the project. 
> The program will timeout after 51 messages and attempt to close the session. 
> It eventually locks in this line: 
>             this.connection.syncSendPacket(info); 
> in the rollback method of the TransactionContext class.
> We believe that the most sensible solution to this is that the rollback should respond
to the same timeout as the connection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message