activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Yaussy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQ-771) org.apache.activemq.broker.TransportConnection::stop should not attempt to send a message over the connection.
Date Wed, 16 Aug 2006 12:38:23 GMT
    [ https://issues.apache.org/activemq/browse/AMQ-771?page=comments#action_36794 ] 
            
Kevin Yaussy commented on AMQ-771:
----------------------------------

Rob,

The fixes I have for the 4.0.1 release for this issue rely upon the fix I made to DemandForwardingBridgeSupport
for issue [ https://issues.apache.org/activemq/browse/AMQ-776?page=all ].

So, for 4.0.2, as I reported in AMQ-776, the 4.0.1 fix did not fix 4.0.2.  I will have to
try and track down what is wrong there and get 4.0.2 fixed for AMQ-776.

At any rate, I should describe this issue better:

Not only is there an issue with TransportConnection::stop, wherein it attempts to send something
on the socket before closing, there are problems in general with the fact that FailoverTransport
is decorated by MutexTransport.  When publishing to a consumer between two brokers, and the
consumer-side broker is frozen, and the socket fills up, then the FailoverTransport (InactivityMonitor)
attempts to close down the connection.  This will fail, as everything is blocked around the
MutexTransport.  See the scenario list below for how to recreate the problem.  

The changes I made are rather surgical, in order to make it work.  I wasn't particularly happy
with them, but maybe they are acceptable.  I will attach patches as soon as I get AMQ-776
working.  But, the changed source files include:
org.apache.activemq.transport.MutexTransport
org.apache.activemq.transport.failover.FailoverTransport
org.apache.activemq.transport.tcp.TcpTransport
org.apache.activemq.broker.TransportConnection

The changes were not tested against all unit tests, so there may be similar changes required
to other files (i.e. some other transport than TcpTransport).


Scenario (using ConsumerTool and ProducerTool from examples):
-Start broker A
-Start broker B
-Start consumer, on FOO, attaching to broker B (failover transport, only broker B)
-Start publisher, on FOO, publishing large messages, such as 10K bytes, attaching to broker
A (failover transport, only broker A)
-On Solaris, pstop broker B

Wait for the socket to fill up, and then when broker A reports the dead connection, notice
that it does not close off the connection properly.  Do a kill-3 on broker A and note that
it is waiting on MutexTransport lock and FailoverTransport can't close off the connection.


> org.apache.activemq.broker.TransportConnection::stop should not attempt to send a message
over the connection.
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-771
>                 URL: https://issues.apache.org/activemq/browse/AMQ-771
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Connector
>    Affects Versions: 4.0.1, 4.0
>            Reporter: Kevin Yaussy
>         Assigned To: Rob Davies
>
> Especially when using "failover", there can be a problem with respect to TransportConnection::stop
attempting to send a "shutdown" message over the connection.  If another thread is sending
messages to the connection, and it gets stuck for some reason, such as a network freeze, the
target machine panics, or the target process freezes for some reason, the TransportConnection::dispatch
will eventually block, locking the MutextTransport object.  When the InactivityMonitor wakes
up and detects that the connection is dead, it will go through the process of stopping the
connection.  This goes back into TransportConnection, and calls stop, which attemtps to lock
the MutexTransport so it can send the "shutdown" command.  Now, both threads are stuck, potentially
for a long time, as a box panic will not cleanly close the tcp connection.
> I'm not sure the rationale for wanting to send a shutdown command to the other side of
the connection, since the target has to handle the connection going down hard anyway.  Seems
to me, if you are intending on closing the connection, just close it - don't try to be nice
to the other side.  Especially in this code path, there is something wrong with the other
side anyway.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/activemq/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message