activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: org.apache.activemq.broker.TransportConnection.Transport - Transport Connection to: tcp://XX.XX.XXX.XXX:5445 failed: java.io.EOFException
Date Sat, 15 Apr 2017 02:07:13 GMT
An EOFException is an indication that the OS believes that the socket is
closed, so at the socket level, the OS will not hold onto large numbers of
sockets under the conditions you describe.  You can confirm this by using
netstat to see how many sockets are open when you reach (or approach) the
failure point.

At the application level (i.e. the Java JVM), it's unlikely that the thread
can't be destroyed due to the exact reason you described (that it can't
tell that the socket is dead), but it's possible that it's not able to exit
due to some other reason. You can check whether this theory might hold
water by seeing how many threads are alive soon after you restart the
broker and comparing that with the number of threads alive just before you
restart the broker the next time. If you have JMX enabled, JConsole or
JVisualVM can both do that, or you can use the jstack command line tool
from a JVM.

Does the OutOfMemoryError indicate that heap space is the resource you ran
out of, or was it something else (threads, file handles, etc.)? If it was
heap, you can use a memory profiler or sampler to see what objects are
being created, which might help someone on this list figure out what's
going on.

Also, did you configure appropriate limits for the memory store so producer
flow control will prevent you from filling memory just from accepting too
many non-persistent messages (or paging in too many messages from a
persistence store)? And is your heap large enough for the way you're using
your broker?

Tim

On Apr 13, 2017 7:34 AM, "Siebo" <blacktorai13@gmail.com> wrote:

I'm facing the same problem and after 2-3 days, ActiveMQ stopped because of
OutOfMemoryError on broker side:



After doing some research, I have some theories:
- TCP connection with EOFException cannot be closed.
- Each TCP connection was run under a separate thread, so the thread cannot
be destroyed due to the TCP connection still alive.
- When the number of threads exceed limitation, it caused OutOfMemoryError.

The problem occurred from last 3 months and I have to restart ActiveMQ
service daily to avoid OutOfMemoryError.
I appreciate all of your ideas and solutions.

Note: I am using ActiveMQ 5.14.1

Thanks,
Siebo



--
View this message in context: http://activemq.2283324.n4.
nabble.com/org-apache-activemq-broker-TransportConnection-Transport-
Transport-Connection-to-tcp-XX-XX-XXX-XXX-5n-tp4722840p4724925.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message