activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin C." <mart...@gmx.at>
Subject Failed network bridge could not be re-established: "Cannot send, channel already failed"
Date Wed, 17 Aug 2011 08:44:29 GMT
Hi,

we use a network-of-broker configuration with a central broker and a
lot of hubs, all using ActiveMQ 5.5.0. Today, one of the bridges
collapsed and could not be re-established. Below you see the log-file.
You can see, that first a client connection failed with "Cannot send,
channel has already failed", a minute later the bridge collapsed and
could not be re-established until restarting the broker on the hub.
After some time, it also starts failing with the "Cannot send, channel
has already failed" error. Any hints on what could cause this issue?


Configuration:
--------------
  <networkConnectors>
      <!-- see http://activemq.2283324.n4.nabble.com/Fwd-Saving-bandwith-in-a-network-of-brokers-with-lots-of-brokers-via-WAN-tp3635292p3635356.html
           for the reasons for the destination filter and its odd syntax -->
      <networkConnector name="siteToCentral"
        uri="static:(ssl://central:61617)?maxReconnectDelay=5000"
        duplex="true"
        conduitSubscriptions="true"
        destinationFilter="Queue.central.>,ActiveMQ.Advisory.Consumer.Queue.site.>,ActiveMQ.Advisory.Consumer.Topic.central.>,ActiveMQ.Advisory.Consumer.Topic.site.>">
        <dynamicallyIncludedDestinations>
          <!-- queues -->
          <queue physicalName="central.>" />
          <queue physicalName="site.>" />
          <!-- topics -->
          <topic physicalName="central.>" />
          <topic physicalName="site.>" />
        </dynamicallyIncludedDestinations>
      </networkConnector>
    </networkConnectors>


LOG-FILE:
---------

2011-08-17 08:12:54,132 | INFO  | Transport failed:
org.apache.activemq.transport.InactivityIOException: Cannot send,
channel has already failed: 192.168.0.4/192.168.0.4:36012 |
org.apache.activemq.broker.TransportConnection.Transport | Async
Exception Handler

[...]

2011-08-17 08:13:26,591 | INFO  | localhost bridge to localhost
stopped | org.apache.activemq.network.DemandForwardingBridgeSupport |
ActiveMQ Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:26,599 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8234
2011-08-17 08:13:26,600 | INFO  | Transport failed:
java.io.EOFException |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
Transport: ssl:///192.168.0.26:44196
2011-08-17 08:13:28,442 | WARN  | Network connection between
vm://localhost#116 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.
DemandForwardingBridgeSupport | ActiveMQ Transport:
ssl://central/192.168.1.5:61617
2011-08-17 08:13:28,451 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:30,412 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8243
2011-08-17 08:13:32,525 | WARN  | Network connection between
vm://localhost#118 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:33,481 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:36,161 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8250
2011-08-17 08:13:37,109 | WARN  | Network connection between
vm://localhost#120 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:37,137 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:41,812 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8257
2011-08-17 08:13:43,984 | WARN  | Network connection between
vm://localhost#122 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:44,013 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:49,423 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8264
2011-08-17 08:13:50,374 | WARN  | Network connection between
vm://localhost#124 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:50,379 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:55,380 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8271
2011-08-17 08:13:55,485 | WARN  | Network connection between
vm://localhost#126 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:13:55,523 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:00,552 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8278
2011-08-17 08:14:00,664 | WARN  | Network connection between
vm://localhost#128 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:00,667 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:05,668 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8285
2011-08-17 08:14:05,785 | WARN  | Network connection between
vm://localhost#130 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:05,788 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:10,789 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8292
2011-08-17 08:14:10,906 | WARN  | Network connection between
vm://localhost#132 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:10,909 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:15,910 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8299
2011-08-17 08:14:16,026 | WARN  | Network connection between
vm://localhost#134 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:16,029 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:21,029 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8306
2011-08-17 08:14:21,081 | WARN  | Network connection between
vm://localhost#136 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:21,081 | WARN  | Network connection between
vm://localhost#136 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: org.apache.activemq.transport.InactivityIOException:
Cannot send, channel has already failed: central/192.168.1.5:61617 |
org.apache.activemq.network.DemandForwardingBridgeSupport |
StartRemotelBridge: localBroker=vm://localhost#136
2011-08-17 08:14:22,049 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:27,049 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8314
2011-08-17 08:14:27,165 | WARN  | Network connection between
vm://localhost#138 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:27,168 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:32,787 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8321
2011-08-17 08:14:32,895 | WARN  | Network connection between
vm://localhost#140 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:32,898 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Task-8324
2011-08-17 08:14:37,896 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8325
2011-08-17 08:14:38,005 | WARN  | Network connection between
vm://localhost#142 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:38,008 | INFO  | localhost bridge to Unknown stopped
| org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
2011-08-17 08:14:43,009 | INFO  | Establishing network connection from
vm://localhost?async=false&network=true to ssl://central:61617 |
org.apache.activemq.network.DiscoveryNetworkConnector | Simple
Discovery Agent-8335
2011-08-17 08:14:43,127 | WARN  | Network connection between
vm://localhost#144 and ssl://central/192.168.1.5:61617 shutdown due to
a remote error: java.io.IOException: Unexpected error occured |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://central/192.168.1.5:61617
...


Thanks in advance,
Martin

Mime
View raw message