activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From TimDog <>
Subject Re: HTTP Broker Transport Holding Dead Connections
Date Sat, 15 Sep 2007 04:03:15 GMT

Actually for a broker connection, RegionBroker is the class that's refusing
to allow the connection because of the duplicate clientID and zombie
connection.  Interesting.  I'm going to have to think of a clever way to
shutdown the previous dead connection.   I implemented a quick fix and it
works, but I think there is a better way.  Something tells me that this post
will probably be better placed in the "Developer" section. 

Well, seeing as how this message went unanswered for so long, I decided to
crack open the source.  Might as well help all those who might have a
similar question in the future.

I'm going to call it a bug.  There's a Map in HttpTunnelServlet that maps
clientID's to BlockingQueue transport objects.  If a normal client stops
without notifying the server(which most clients do) it maintains a zombie
object referenced by this clientID.  When the client connects again, it has
a new clientId which is generated with a unique time.  That's why it's
unnoticeable with normal http client connections.  However, the map
continues to store these objects and I can see that this might(will,
actually) lead to a memory leak.

The problem becomes immediately apparent when using the http transport to
establish a network of brokers.  The clientId of a broker client is fixed
(by the name or server name).  If the broker client fails, the server never
clears this object out of the Map.  When trying to reconnect, it sees that
it has a valid object in the map and refuses the new broker client
connection.  The only way to get the client broker to connect is to change
it's name...and there's still the zombie object in the Map.

I don't see any mechanism to test for the validity of the connection.(Some
timer to say, "have I heard from this client this connection
still alive.)  It seems at the very least if a new client connection comes
in with the same clientId...the old one should be destroyed.  Also, there
should be some  process to kill the items in the map.

I'm going to try to build the source and implement a fix.  I thought I'd
share, just in case someone else might have a problem in the future.

TimDog wrote:
> I am using AMQ 4.1.1.  I've set the network connectors between brokers to
> use the http protocol.  Everything works great as far as transport of
> messages.
> Let's say I have two brokers mutually connected over HTTP.  Broker A and
> Broker B.
> Broker A goes down down unexpectedly.(A non-graceful shutdown)   When it
> tries to reconnect to Broker B, I get the following message.(Addresses
> changed to protect the innocent)
> - Network connection
> between vm://brokerA#12 and HTTP Reader http://brokerB:80 shutdown due to
> a remote error: javax.jms.InvalidClientIDException: Broker: brokerB -
> Client: NC_BrokerA_outboundBrokerA already connected
> It appears that the broker B never releases the connection.  When the
> broker A tries to reconnect, broker B thinks it already has a connection
> to Broker A.
> This doesn't seem to happen over the TCP transport.  Only the HTTP
> transport.  My guess is that during a normal shutdown, some command comes
> over the wire to tell Broker B that Broker A is going down and releases
> the connection.
> Is there a configuration for connection Timeout that I can set for the
> transports?  Or a way to get around this issue, so that I don't have to
> manually kill the HTTP connections in JMS?  Any thoughts appreciated.
> Thanks!
> Tim

View this message in context:
Sent from the ActiveMQ - User mailing list archive at

View raw message