activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Torsten Mielke (JIRA)" <>
Subject [jira] [Commented] (AMQ-5692) Inactivity monitor does not time out on stuck socket writes
Date Tue, 07 Apr 2015 11:58:12 GMT


Torsten Mielke commented on AMQ-5692:

Attached reproducer in 
To reproduce issue follow these steps:

- Run broker with stomp connector on port 61613 and this connector configuration
<transportConnector name="stomp"  uri="stomp://,0&amp;transport.useKeepAlive=true&amp;trace=true"/>
- Run consumer 
perl ./ consumer

- Run producer 
perl ./ producer

After consuming around 8000 messages, the consumer hangs and won't report any more messages
Take a broker thread dump and observe two stuck threads as in (1) below.
Despite the configured stomp heart-beat defaultHeartBeat=10000,0 the connection does not time
out after 10 seconds and no keep-alive is sent either. 

Have not managed to reproduce the issue with the ActiveMQ stomp client library. 

"ActiveMQ BrokerService[localhost] Task-4" daemon prio=5 tid=0x00007fe9ca087800 nid=0x650b
runnable [0x00000001192d9000]
   java.lang.Thread.State: RUNNABLE
	at Method)
	at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(
	at org.apache.activemq.transport.tcp.TcpTransport.oneway(
	at org.apache.activemq.transport.stomp.StompTransportFilter.sendToStomp(
	at org.apache.activemq.transport.stomp.StompSubscription.onMessageDispatch(
	at org.apache.activemq.transport.stomp.ProtocolConverter.onActiveMQCommand(
	at org.apache.activemq.transport.stomp.StompTransportFilter.oneway(
	at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(
	at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(
	at org.apache.activemq.transport.MutexTransport.oneway(
	at org.apache.activemq.thread.PooledTaskRunner.runTask(
	at org.apache.activemq.thread.PooledTaskRunner$
	at java.util.concurrent.ThreadPoolExecutor.runWorker(
	at java.util.concurrent.ThreadPoolExecutor$

"ActiveMQ Transport: tcp:///" daemon prio=5 tid=0x00007fe9ca086800
nid=0x3b0f waiting on condition [0x0000000118f2d000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007c0ecc2f0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
	at java.util.concurrent.locks.LockSupport.park(
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(
	at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(
	at java.util.concurrent.locks.ReentrantLock.lock(
	at org.apache.activemq.transport.MutexTransport.onCommand(
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(
	at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(
	at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(
	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompAck(
	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(
	at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(
	at org.apache.activemq.transport.TransportSupport.doConsume(
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(

>  Inactivity monitor does not time out on stuck socket writes
> ------------------------------------------------------------
>                 Key: AMQ-5692
>                 URL:
>             Project: ActiveMQ
>          Issue Type: Improvement
>          Components: Broker
>    Affects Versions: 5.11.1
>            Reporter: Torsten Mielke
>              Labels: broker, inactivity
>         Attachments:
> It is possible that a socket write is stuck but the inactivity monitor currently does
not time out on a socketWrite. 
> {}
>     final void writeCheck() {
>         if (inSend.get()) {
>             LOG.trace("Send in progress. Skipping write check.");
>             return;
>         }
> {code}
> As a result a connection that is stuck in a tcp write will never be taken down due to
inactivity. If a client misbehaves the broker will not be able to clear that connection as
part of the inactivity monitoring.
> Now AMQ-2511 introduced a counter on the reachCheck() to detect it a socket read in progress
really retrieves data or is stuck. 
> I propose for a similar mechanism being applied on the writeCheck() operation so that
a socket write that is stuck can be detected and the connection can be closed.

This message was sent by Atlassian JIRA

View raw message