Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: pass (nike.apache.org: domain of kai.hackemesser@gmail.com
 designates 209.85.217.171 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <E1BC81D6-5D8D-49DC-922F-B8E9AA636D00@fusesource.com>
References: 
 <CAMnNrd_zkL3-5saj1HO=kOwQDkUkjT3+w9TgnJSZN7sdi0=yXQ@mail.gmail.com>
	<E1BC81D6-5D8D-49DC-922F-B8E9AA636D00@fusesource.com>
Date: Fri, 3 Feb 2012 11:21:03 +1300
Message-ID: 
 <CAMnNrd_UQSjRZXgQXe8o422Y-pWO1-4YkCpcqvVbk7zMd=M0ug@mail.gmail.com>
Subject: Re: static network connection lost and not recreated
From: Kai Hackemesser <kai.hackemesser@gmail.com>
To: users@activemq.apache.org
Content-Type: multipart/alternative; boundary=bcaec555543093bd8904b8029c3f

--bcaec555543093bd8904b8029c3f
Content-Type: text/plain; charset=UTF-8

Hi

The symptoms of our configuration got more and more weird. We use currently
topics where we have up to 5 producers and about the same number of
consumers, mostly spread over the two boxes having a broker each. We did a
run overnight and it ended with one broker falling over with not enough
memory (our fault)) but the other one is in a very inconsistent state. It
doesn't accept incoming connections from the other box (the clients are
configured to try first the local then the other broker) and I can see very
high numbers of subscribers on one of the topics (a value of up to 4 would
make sense but JMX currently shows 314 - sometimes the value goes up to
above 3000 for a second). the clients on the remote box loop in message
like:

INFO [02 Feb 2012 21:33:36,855]: Successfully reconnected to
tcp://otherbox:61616
WARN [02 Feb 2012 21:33:36,855]: Transport (otherbox/10.100.88.17:61616)
failed to tcp://kd-sage3:61616 , attempting to automatically reconnect due
to: java.net.SocketException: Socket closed

On the broker side I got:

2012-02-02 21:33:36,858 | INFO  | Transport failed: java.io.EOFException |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
Transport: tcp:///10.100.88.16:36006
2012-02-02 21:33:36,914 | INFO  | Transport failed:
java.net.SocketException: Broken pipe |
org.apache.activemq.broker.TransportConnection.Transport | Async Exception
Handler

We use a Spring connection factory to connect, which is set to reconnect
automatically if connection is lost:

    @Bean
    public CachingConnectionFactory cachingConnectionFactory() {
        CachingConnectionFactory factory = new CachingConnectionFactory();
        factory.setReconnectOnException(true);
        factory.setTargetConnectionFactory(jmsConnectionFactory());
        return factory;
    }

    @Bean
    public ActiveMQConnectionFactory jmsConnectionFactory() {
        ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory();

factory.setBrokerURL("failover:(tcp://localhost:61616,tcp://otherbox:61616)?randomize=false");
        return factory;
    }

The only solution we found was to restart the broker, but as this stuff is
planned to go into production we want to serve them a configuration that
can have the broker running for months without need to interfere with it.

Cheers,
Kai


2012/2/2 Torsten Mielke <torsten@fusesource.com>

> Hi,
>
> This should generally work alright.
> You may want to configure for debug logging on org.apache.activemq.network
> and inspect the logging output carefully when the bridge should get
> restored.
>
>
> Torsten Mielke
> torsten@fusesource.com
> tmielke@blogspot.com
>
>
> On Feb 2, 2012, at 1:57 AM, Kai Hackemesser wrote:
>
> > Hello,
> >
> > I have set up a static NC between two brokers of Version 5.5.1, and we
> have
> > now seen twice within our preproduction test that the connection was lost
> > and not reestablished automatically. We could not establish it through
> JMX
> > using the NC's start operation but had to restart one or both brokers. Is
> > this a known bug or a  configuration issue?
> >
> > Cheers,
> > Kai
>
>

--bcaec555543093bd8904b8029c3f--