activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: org.apache.activemq.broker.TransportConnection.Transport - Transport Connection to: tcp://XX.XX.XXX.XXX:5445 failed: java.io.EOFException
Date Wed, 19 Apr 2017 13:10:26 GMT
Siebo,

On Apr 18, 2017 4:28 AM, "Siebo" <blacktorai13@gmail.com> wrote:

Hi Tim,


> OK, for the EOFException, are your brokers behind a load balancer like
> this thread's OP was? It sounds like you're not, so what's on the other
> end of those connections? One possibility is real client processes, or
> another is another broker in a network of brokers setup. In either case,
> are all clients/brokers misbehaving equally, or are some fine and others
> very unhappy?

→ I can tell that all consumers were working fine while EOFException
occurred continuously. Pretty weird huh?


My definition of misbehaving didn't have to mean that the client would
report errors. For example, if the client was continually opening new
connections to the broker without closing the old ones, it would be
misbehaving (not doing what it's supposed to) even though it would
successfully process data and would report no errors.

So are all client IPs represented proportionally in the EOFExceptions?

> The JIRA you linked to was specifically related to websockets; is that a
> configuration you're using?

→ I can't really get your idea. I made no configuration to ActiveMQ's
activemq.xml other than using MySQL persistenceAdapter instead of kahaDB. So
current configuration is using websockets I think.


You would have to explicitly enable websockets, and your clients would have
to explicitly use them, so it sounds like you're not using them. That's
fine, it just means that JIRA doesn't apply to you.

> I completely understand not being able to upgrade the version of ActiveMQ
> on a production server, but I don't buy the argument that it's not
> possible to take a thread dump just because it's a production server.
> Taking a thread dump is not a performance impact, and you should push back
> on whoever is telling you that you're not allowed to do it. Unless, of
> course, you don't care about the thread count and only want to pursue the
> EOFException question.

→ Yes, you're right. My problem is having no right to generate thread dump
on production server. It had already been requested. I hope to receive the
thread dump soon...


I misunderstood: I thought you were saying that your request was denied,
but now I understand that you just needed someone else to perform it. No
problem there.

> Also, what does the stack trace for the EOFException say the broker was
> doing when the EOFException occurred?

→ The broker did nothing in this case I think. It just started and the
EOFException appeared in log file and no message was sent into it (on
'inactive' ActiveMQ server).
As I know, ActiveMQ InactivityMonitor would check for 'inactive' connection
all the time. I wonder if it is the only job that broker was doing while
EOFException occurred.
Here are some lines of log, the EOFException was log about once a min:


Your attachment contains stacks for three threads. The first is a failover
transport that is trying to reconnect because the connection has been lost.
The third is a TCP transport that is waiting for data to be sent to it. The
second is a thread in a thread pool that has not been given any work to do;
thats not a problem, it just means you have some extra capacity in your
thread pool (and that's a good thing). None of these threads look like a
problem, though I'm curious why the failover transport isn't connected and
needs to reconnect.

How many threads fall into each of the three categories?

Keep in mind that the broker starts threads when clients connect (to read
data from the sockets), so having more threads on the active broker isn't
entirely unexpected. But the size of the difference might indicate that
clients are opening connections for each message or something similar.
Since you haven't told us anything about your client workload, I can't say
for sure whether the the difference in thread count is expected or is a
cause for concern.

Thanks and best regards,
Siebo



--
View this message in context: http://activemq.2283324.n4.nab
ble.com/org-apache-activemq-broker-TransportConnection-Transport-Transport-
Connection-to-tcp-XX-XX-XXX-XXX-5n-tp4722840p4725022.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message