activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Hill <sh...@futurelogix.co.uk>
Subject Broken Pipe Connecting
Date Thu, 31 Aug 2017 20:18:37 GMT
We are running ActiveMQ in Master/Slave configuration.  Currently 
mqarms02p is the Master.  Earlier today we ran into a situation where we 
lost access to ActiveMQ and on reviewing the client side logs we see the 
information below cycle continuously.

Is there anything we can enable on the server side to better log what 
happened from the server perspective?  Unfortunately in this instance 
the logs had already rolled so we could not see anything, however from 
previously experience we know almost nothing shows up.

The server was online and other clients were using it without incident

Any ideas as to what happened and how we can prevent this occurring in 
future?

08-31 11:46:02 DEBUG - Attempting 0th connect to: tcp://mqarms01p:61616
08-31 11:46:02 TRACE - Starting connection check task for: 
tcp://mqarms01p:61616
08-31 11:46:02 DEBUG - Connect fail to: tcp://mqarms01p:61616, reason: 
java.net.ConnectException: Connection refused
08-31 11:46:02 TRACE - Stopping connection check task for: 
tcp://mqarms01p:61616
08-31 11:46:02 DEBUG - Stopping transport tcp://mqarms01p:61616
08-31 11:46:02 DEBUG - Initialized TaskRunnerFactory[ActiveMQ Task] 
using ExecutorService: 
java.util.concurrent.ThreadPoolExecutor@1f0f564[Running, pool size = 0, 
active threads = 0, queued tasks = 0, completed tasks = 0]
08-31 11:46:02 TRACE - Execute[ActiveMQ Task] runnable: 
org.apache.activemq.transport.tcp.TcpTransport$1@727e793a
08-31 11:46:02 TRACE - Created thread[ActiveMQ Task-1]: Thread[ActiveMQ 
Task-1,5,main]
08-31 11:46:02 TRACE - Closing socket Socket[unconnected]
08-31 11:46:02 DEBUG - Closed socket Socket[unconnected]
08-31 11:46:02 DEBUG - Forcing shutdown of ExecutorService: 
java.util.concurrent.ThreadPoolExecutor@1f0f564[Running, pool size = 1, 
active threads = 0, queued tasks = 0, completed tasks = 1]
08-31 11:46:02 TRACE - Shutdown of ExecutorService: 
java.util.concurrent.ThreadPoolExecutor@1f0f564[Shutting down, pool size 
= 1, active threads = 0, queued tasks = 0, completed tasks = 1] is 
shutdown: true and terminated: false.
08-31 11:46:02 DEBUG - Attempting 0th connect to: tcp://mqarms02p:61616
08-31 11:46:02 TRACE - Starting connection check task for: 
tcp://mqarms02p:61616
08-31 11:46:02 DEBUG - Sending: WireFormatInfo { version=12, 
properties={MaxFrameSize=9223372036854775807, CacheSize=1024, 
ProviderVersion=5.14.3, CacheEnabled=true, ProviderName=ActiveMQ, 
SizePrefixDisabled=false, PlatformDetails=JVM: 1.7.0_80, 24.80-b11, 
Oracle Corporation, OS: Linux, 2.6.32-358.6.2.el6.x86_64, amd64, 
MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true, 
MaxInactivityDuration=30000, TightEncodingEnabled=true, 
StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
08-31 11:46:02 TRACE - Starting connection check task for: 
tcp://mqarms02p/172.16.101.116:61616@47626
08-31 11:46:02 TRACE - TCP consumer thread for 
tcp://mqarms02p/172.16.101.116:61616@47626 starting
08-31 11:46:02 TRACE - Stopping connection check task for: 
tcp://mqarms02p/172.16.101.116:61616@47626
08-31 11:46:02 DEBUG - Using min of local: WireFormatInfo { version=12, 
properties={MaxFrameSize=9223372036854775807, CacheSize=1024, 
ProviderVersion=5.14.3, CacheEnabled=true, ProviderName=ActiveMQ, 
SizePrefixDisabled=false, PlatformDetails=JVM: 1.7.0_80, 24.80-b11, 
Oracle Corporation, OS: Linux, 2.6.32-358.6.2.el6.x86_64, amd64, 
MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true, 
MaxInactivityDuration=30000, TightEncodingEnabled=true, 
StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]} and remote: 
WireFormatInfo { version=12, properties={MaxFrameSize=104857600, 
CacheSize=1024, ProviderVersion=5.14.3, CacheEnabled=true, 
ProviderName=ActiveMQ, SizePrefixDisabled=false, 
MaxInactivityDurationInitalDelay=10000, PlatformDetails=JVM: 1.8.0_111, 
25.111-b15, Oracle Corporation, OS: Linux, 3.8.13-68.3.3.el6uek.x86_64, 
amd64, TcpNoDelayEnabled=true, MaxInactivityDuration=30000, 
TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
08-31 11:46:02 DEBUG - Received WireFormat: WireFormatInfo { version=12, 
properties={MaxFrameSize=104857600, CacheSize=1024, 
ProviderVersion=5.14.3, CacheEnabled=true, ProviderName=ActiveMQ, 
SizePrefixDisabled=false, MaxInactivityDurationInitalDelay=10000, 
PlatformDetails=JVM: 1.8.0_111, 25.111-b15, Oracle Corporation, OS: 
Linux, 3.8.13-68.3.3.el6uek.x86_64, amd64, TcpNoDelayEnabled=true, 
MaxInactivityDuration=30000, TightEncodingEnabled=true, 
StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
08-31 11:46:02 DEBUG - tcp://mqarms02p/172.16.101.116:61616@47626 before 
negotiation: OpenWireFormat{version=12, cacheEnabled=false, 
stackTraceEnabled=false, tightEncodingEnabled=false, 
sizePrefixDisabled=false, maxFrameSize=9223372036854775807}
08-31 11:46:02 DEBUG - tcp://mqarms02p/172.16.101.116:61616@47626 after 
negotiation: OpenWireFormat{version=12, cacheEnabled=true, 
stackTraceEnabled=true, tightEncodingEnabled=true, 
sizePrefixDisabled=false, maxFrameSize=104857600}
08-31 11:46:02 DEBUG - conn: ID:tnsags01p.xxxx.com-53927-1504137796960-5:1
08-31 11:46:02 DEBUG - session: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1441358
08-31 11:46:02 DEBUG - restore consumer: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1441358:1 in pull mode 
pending recovery, overriding prefetch: 1000
08-31 11:46:02 DEBUG - consumer: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1441358:1
08-31 11:46:02 DEBUG - session: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:-1
08-31 11:46:02 DEBUG - restore consumer: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:-1:1 in pull mode pending 
recovery, overriding prefetch: 1000
08-31 11:46:02 DEBUG - consumer: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:-1:1
08-31 11:46:02 DEBUG - session: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1442703
08-31 11:46:02 DEBUG - producer: 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1442703:1
08-31 11:46:02 DEBUG - Connection established
08-31 11:46:02 INFO  - Successfully reconnected to tcp://mqarms02p:61616
08-31 11:46:02 TRACE - Running task iteration 0 - 
org.apache.activemq.transport.failover.FailoverTransport$2@112605fd
08-31 11:46:02 TRACE - Run task done: 
org.apache.activemq.transport.failover.FailoverTransport$2@112605fd
08-31 11:46:02 TRACE - Run task done: 
org.apache.activemq.transport.failover.FailoverTransport$2@112605fd
08-31 11:46:02 TRACE - Received ConnectionControl: ConnectionControl 
{commandId = 0, responseRequired = false, suspend = false, resume = 
false, close = false, exit = false, faultTolerant = false, 
connectedBrokers = , reconnectTo = , token = null, rebalanceConnection = 
false}
08-31 11:46:02 DEBUG - Send oneway attempt: 17752 failed for command: 
ActiveMQObjectMessage {commandId = 7234566, responseRequired = true, 
messageId = ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1442703:1:1, 
originalDestination = null, originalTransactionId = null, producerId = 
ID:tnsags01p.xxxx.com-53927-1504137796960-5:1:1442703:1, destination = 
queue://routing_mia_client, transactionId = null, expiration = 0, 
timestamp = 1504192175689, arrival = 0, brokerInTime = 0, brokerOutTime 
= 0, correlationId = null, replyTo = null, persistent = true, type = 
null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId 
= null, compressed = false, userID = null, content = 
org.apache.activemq.util.ByteSequence@1558c374, marshalledProperties = 
null, dataStructure = null, redeliveryCounter = 0, size = 0, properties 
= null, readOnlyProperties = true, readOnlyBody = true, droppable = 
false, jmsXGroupFirstForConsumer = false}
08-31 11:46:02 TRACE - tcp://mqarms02p:61616 handleTransportFailure: 
java.net.SocketException: Broken pipe
java.net.SocketException: Broken pipe
         at java.net.SocketOutputStream.socketWrite0(Native Method)
         at 
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
         at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
         at 
org.apache.activemq.transport.tcp.TcpBufferedOutputStream.write(TcpBufferedOutputStream.java:96)
         at java.io.DataOutputStream.write(DataOutputStream.java:107)
         at 
org.apache.activemq.openwire.v12.BaseDataStreamMarshaller.tightMarshalByteSequence2(BaseDataStreamMarshaller.java:431)
         at 
org.apache.activemq.openwire.v12.MessageMarshaller.tightMarshal2(MessageMarshaller.java:182)
         at 
org.apache.activemq.openwire.v12.ActiveMQMessageMarshaller.tightMarshal2(ActiveMQMessageMarshaller.java:89)
         at 
org.apache.activemq.openwire.v12.ActiveMQObjectMessageMarshaller.tightMarshal2(ActiveMQObjectMessageMarshaller.java:89)
         at 
org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:235)
         at 
org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:193)
         at 
org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:335)
         at 
org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:317)
         at 
org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:94)
         at 
org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:116)
         at 
org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:668)


Mime
View raw message