activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Rodriguez <srodrig...@gmail.com>
Subject Network of brokers --> excessive reset of the connections?
Date Thu, 06 Oct 2011 05:56:19 GMT
Hi all,
I'm using ActiveMQ 5.4.0, in a network of broker configuration, with duplex
connections to send notifications in a server / clients architecture. I was
looking through the logs for an issue I had on site, and I noticed a lot of
warnings and errors related to connection between the members of the network
of brokers.
I've not been able to get enough information on what they mean and would
like to see if I can get some information?
The messages are as follow:

### start log ###
2011-10-04 16:42:04,433 | WARN  | *Network connection between
vm://SRV-01#162 and tcp:///10.20.2.6:59104 shutdown due to a remote error:
java.io.EOFException* | org.apache.activemq.network.DemandForwardingBridge |
ActiveMQ Transport: tcp:///10.20.2.6:59104
2011-10-04 16:42:04,448 | INFO  | SRV-01 bridge to SRV-02 stopped |
org.apache.activemq.network.DemandForwardingBridge | ActiveMQ Task
2011-10-04 16:42:24,729 | WARN  | *Network connection between vm://SRV-01#0
and tcp://SRV-02/10.20.2.6:31616 shutdown due to a remote error:
java.net.SocketException: Connection reset* |
org.apache.activemq.network.DemandForwardingBridge | ActiveMQ Transport:
tcp://SRV-02/10.20.2.6:31616
2011-10-04 16:42:24,807 | INFO  | Establishing network connection from
vm://SRV-01?async=false&network=true to tcp://SRV-02:31616 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent: java.util.concurrent.ThreadPoolExecutor$Worker@3c0965
2011-10-04 16:42:25,837 | INFO  | *Transport failed:
org.apache.activemq.transport.TransportDisposedIOException: Peer
(vm://SRV-01#347) disposed.* |
org.apache.activemq.broker.TransportConnection.Transport | Simple Discovery
Agent: java.util.concurrent.ThreadPoolExecutor$Worker@3c0965
org.apache.activemq.transport.TransportDisposedIOException: Peer
(vm://SRV-01#347) disposed.
at org.apache.activemq.transport.vm.VMTransport.stop(VMTransport.java:159)
 at
org.apache.activemq.transport.vm.VMTransportServer$1.stop(VMTransportServer.java:81)
at
org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
 at
org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
at
org.apache.activemq.transport.ResponseCorrelator.stop(ResponseCorrelator.java:132)
 at org.apache.activemq.util.ServiceSupport.dispose(ServiceSupport.java:43)
at
org.apache.activemq.network.DiscoveryNetworkConnector.onServiceAdd(DiscoveryNetworkConnector.java:137)
 at
org.apache.activemq.transport.discovery.simple.SimpleDiscoveryAgent$1.run(SimpleDiscoveryAgent.java:164)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2011-10-04 16:42:25,837 | WARN  | Could not start network bridge between:
vm://SRV-01?async=false&network=true and: tcp://SRV-02:31616 due to:
java.net.ConnectException: Connection refused: connect |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent: java.util.concurrent.ThreadPoolExecutor$Worker@3c0965
2011-10-04 16:42:25,837 | INFO  | Network connection between vm://SRV-01#346
and tcp://SRV-02:31616 shutdown due to a local error:
org.apache.activemq.transport.TransportDisposedIOException: Peer
(vm://SRV-01#346) disposed. |
org.apache.activemq.network.DemandForwardingBridge | ActiveMQ Task
### end log###

Then after several (fast) retries, the connection is established:

### start log ###
2011-10-04 16:42:34,401 | INFO  | Network connection between vm://SRV-01#354
and tcp://SRV-02/10.20.2.6:31616(SRV-02) has been established. |
org.apache.activemq.network.DemandForwardingBridge | StartLocalBridge:
localBroker=vm://SRV-01#354
2011-10-04 16:45:10,904 | INFO  | Transport failed:
java.net.SocketException: Connection reset |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
Transport: tcp:///10.20.2.12:52293
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
 at
org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:575)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58)
 at
org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:560)
at java.io.DataInputStream.readInt(Unknown Source)
 at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:269)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:226)
 at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:218)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201)
 at java.lang.Thread.run(Unknown Source)
2011-10-04 16:46:32,479 | INFO  | Transport failed:
java.net.SocketException:* Software caused connection abort: recv
failed |*org.apache.activemq.broker.TransportConnection.Transport |
ActiveMQ
Transport: tcp:///10.20.2.13:55160
java.net.SocketException: Software caused connection abort: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
 at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
at
org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:575)
 at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58)
at
org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:560)
 at java.io.DataInputStream.readInt(Unknown Source)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:269)
 at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:226)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:218)
 at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201)
at java.lang.Thread.run(Unknown Source)
### end log ###

Then after a number of loops like this, the broker resets

### start log ###
2011-10-04 17:23:54,256 | INFO  | ActiveMQ Message Broker (SRV-01,
ID:SRV-01-50868-1317764432687-0:0) is shutting down |
org.apache.activemq.broker.BrokerService | Thread-20
2011-10-04 17:23:54,256 | INFO  | ActiveMQ Message Broker (SRV-01,
ID:SRV-01-50868-1317764432687-0:0) is shutting down |
org.apache.activemq.broker.BrokerService | ActiveMQ ShutdownHook
2011-10-04 17:23:54,256 | INFO  | SRV-01 bridge to SRV-02 stopped |
org.apache.activemq.network.DemandForwardingBridge | Thread-20
2011-10-04 17:23:54,256 | INFO  | Network Connector HA Queue Stopped |
org.apache.activemq.network.NetworkConnector | Thread-20
### end log ###

This happen relatively quite frequently (~30min) which is quite worrying.
Anyone experiencing the same problems? Or has a wild guess of what is
causing the error?
Thanks,

-- 
Sebastien Rodriguez

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message