activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: "allowLinkStealing" as a client configuration
Date Mon, 25 Sep 2017 04:11:41 GMT
I finally got time on a computer (not my phone) to look at this.

First, the reason that you're not seeing a stack trace is our fault, not
the result of the JVM optimizing away the stack trace. If you look at line
858 of
https://github.com/apache/activemq/blob/activemq-5.14.x/activemq-broker/src/main/java/org/apache/activemq/broker/TransportConnection.java,
you'll see that we're including the exception object inline in the message
(which just calls toString() without providing stack trace info) rather
than passing it as a second argument after the message.

Thankfully there's only one place in the code where that error message
occurs, which is
activemq-broker/src/main/java/org/apache/activemq/broker/region/RegionBroker.java
<https://github.com/apache/activemq/blob/master/activemq-broker/src/main/java/org/apache/activemq/broker/region/RegionBroker.java>
line 255. Unfortunately, the null value at that line is the result of
evaluating oldContext.getConnection().getRemoteAddress()), and it's not
immediately obvious why that address either was never set to a non-null
value or was later overwritten with a null value. It's possible that this
null value will turn out to be relevant to your actual question, but for
now let's treat it as unrelated and see if we can solve your primary
question.

Are you able to reproduce this problem in a non-operational environment
where you can attach a debugger? I think the next step here will be to use
a debugger to examine the logic for disconnecting the old connection and
then re-connecting the new one, so if you don't yet have a way to do that,
that would be your next step. Either that, or you could write a unit test
that reproduces the problem against an embedded broker, and then someone
other than you could step through that test with a debugger.

One other question: does the re-connection work if you wait longer than
5-10 seconds? If you wait a minute, or five, does the re-connection
succeed? I'm not suggesting that's the desired behavior, I just want to
better characterize whether this is a timing issue/race condition, or
whether the bad behavior is unrelated to timing.

Tim

On Thu, Sep 21, 2017 at 6:44 AM, khandelwalanuj <
anuj.cool.khandelwal@gmail.com> wrote:

> I can only see warning message in broker logs:
>
> [2017-09-16 14:15:59.540-0400] [ActiveMQ NIO Worker 3]
> [org.apache.activemq.broker.TransportConnection] [WARN]
> [TransportConnection.java:org.apache.activemq.broker.TransportConnection:
> processAddConnection()::858]
> [Failed to add Connection
> id=ID:app1a.prod.c.ia55.net-46199-1505585017494-3:2,
> clientId=integratedapp1a.prod.c.ia55.net_conn_listener due to
> javax.jms.InvalidClientIDException: Broker: mqbrokerbaly - Client:
> integratedapp1a.prod.c.ia55.net_conn_listener already *connected from
> null]
> *
>
> On the client side the exception received is :
> [ 2017-09-16 18:01:59,897 [EngineTriggerSide2MessageListenerContainer-1]
> []
> listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessf
> ul():907
> ERROR]: Could
> not refresh JMS Connection for destination 'topic://RecNotification' -
> retrying in 5000 ms. Cause: Broker: mqbroker- Client:
> integratedapp1a.prod.c.ia55.net_conn_listener already connected from null
> [ 2017-09-16 18:01:59,974 [ActiveMQ Transport:
> tcp://mqbroker/*:*:*:*:61616@57384] []
> failover.FailoverTransport.handleTransportFailure():260  WARN ]: Transport
> (tcp://mqbrok
> er/10.12.19.62:61616@57384) failed, reason:  , attempting to automatically
> reconnect
> java.io.EOFException
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
>         at
> org.apache.activemq.openwire.OpenWireFormat.unmarshal(
> OpenWireFormat.java:258)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.readCommand(
> TcpTransport.java:221)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(
> TcpTransport.java:213)
>         at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
> > When you restart a client, how do you you stop the client (hard crash,
> > kill
> > -9, graceful shutdown, etc.) and how long do you wait before starting it
> > again? Could this be a situation where the broker simply hasn't
> discovered
> > that the previous TCP connection is dead?
> It's a graceful shutdown and client gets restarted quickly (~5 to 10
> seconds) after the stop.
>
>
>
>
> Thanks,
> Anuj
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message