activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Wei (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMQ-4150) in pure master/slave, when the network is not stable, neither master nor slave can provide service to client
Date Wed, 31 Oct 2012 09:11:11 GMT
Wei Wei created AMQ-4150:
----------------------------

             Summary: in pure master/slave, when the network is not stable, neither master
nor slave can provide service to client
                 Key: AMQ-4150
                 URL: https://issues.apache.org/jira/browse/AMQ-4150
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker, Transport
    Affects Versions: 5.7.0
         Environment: CentOS release 5.6 (Final) java version "1.6.0_30"
            Reporter: Wei Wei


We use pure master/slave. When the network between master and slave is not stable, the connection
between master and slave shutdown, then the slave try to startAllConnectors because of the
follow exception:

2012-10-25 18:48:12,083 | ERROR | Network connection between vm://172_17_7_92_61619#0 and
tcp:///172.17.7.91:61618 shutdown: Connection reset | org.apache.activemq.broker.ft.MasterConnector
| ActiveMQ Transport: tcp:///172.17.7.91:61618
java.net.SocketException: Connection reset
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
	at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
	at java.io.DataOutputStream.flush(DataOutputStream.java:106)
	at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:184)
	at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:262)
	at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:244)
	at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
	at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
	at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
	at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
	at org.apache.activemq.broker.ft.MasterConnector.serviceRemoteCommand(MasterConnector.java:268)
	at org.apache.activemq.broker.ft.MasterConnector$2.onCommand(MasterConnector.java:138)
	at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:229)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:222)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:204)
	at java.lang.Thread.run(Thread.java:662)

After the exception the follow log was printed:

2012-10-25 18:48:12,084 | WARN  | Master Failed - starting all connectors | org.apache.activemq.broker.BrokerService
| ActiveMQ Transport: tcp:///172.17.7.91:61618

it seems that the connector started properly, but it can not provide service to client.


After a while, another exception occured:

2012-10-25 18:49:01,212 | ERROR | Network connection between vm://172_17_7_92_61619#0 and
tcp:///172.17.7.91:61618 shutdown: Broken pipe | org.apache.activemq.broker.ft.MasterConnector
| InactivityMonitor Async Task: java.util.concurrent.ThreadPoolExecutor$Worker@351dd9b3
java.net.SocketException: Broken pipe
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
	at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
	at java.io.DataOutputStream.flush(DataOutputStream.java:106)
	at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:184)
	at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:262)
	at org.apache.activemq.transport.AbstractInactivityMonitor.access$500(AbstractInactivityMonitor.java:40)
	at org.apache.activemq.transport.AbstractInactivityMonitor$3.run(AbstractInactivityMonitor.java:151)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)

this exception lead to the slave trying to startAllConnectors again, this time the follow
exception occured:

2012-10-25 18:49:26,233 | ERROR | Failed to startAllConnectors | org.apache.activemq.broker.BrokerService
| InactivityMonitor Async Task: java.util.concurrent.ThreadPoolExecutor$Worker@351dd9b3
java.io.IOException: Transport Connector could not be registered in JMX: Failed to bind to
server socket: nio://0.0.0.0:61619 due to: java.net.BindException: Address already in use
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:27)
	at org.apache.activemq.broker.BrokerService.registerConnectorMBean(BrokerService.java:1836)
	at org.apache.activemq.broker.BrokerService.startTransportConnector(BrokerService.java:2327)
	at org.apache.activemq.broker.BrokerService.startAllConnectors(BrokerService.java:2243)
	at org.apache.activemq.broker.BrokerService.masterFailed(BrokerService.java:446)
	at org.apache.activemq.broker.ft.MasterConnector.shutDown(MasterConnector.java:345)
	at org.apache.activemq.broker.ft.MasterConnector.serviceRemoteException(MasterConnector.java:250)
	at org.apache.activemq.broker.ft.MasterConnector$2.onException(MasterConnector.java:144)
	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
	at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:126)
	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
	at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:101)
	at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:160)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:268)
	at org.apache.activemq.transport.AbstractInactivityMonitor$3.run(AbstractInactivityMonitor.java:154)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Failed to bind to server socket: nio://0.0.0.0:61619 due to:
java.net.BindException: Address already in use
	at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
	at org.apache.activemq.transport.tcp.TcpTransportServer.bind(TcpTransportServer.java:138)
	at org.apache.activemq.transport.tcp.TcpTransportFactory.doBind(TcpTransportFactory.java:60)
	at org.apache.activemq.transport.TransportFactory.bind(TransportFactory.java:131)
	at org.apache.activemq.broker.TransportConnector.createTransportServer(TransportConnector.java:329)
	at org.apache.activemq.broker.TransportConnector.getServer(TransportConnector.java:155)
	at org.apache.activemq.broker.TransportConnector.asManagedConnector(TransportConnector.java:110)
	at org.apache.activemq.broker.BrokerService.registerConnectorMBean(BrokerService.java:1831)
	... 16 more
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind(Native Method)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
	at org.apache.activemq.transport.nio.NIOTransportFactory$2.createServerSocket(NIOTransportFactory.java:69)
	at org.apache.activemq.transport.tcp.TcpTransportServer.bind(TcpTransportServer.java:134)
	... 22 more

At last, neither the master nor the slave can provide service to client.





--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message