activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Harter <harter....@gmail.com>
Subject Memory Leak in DefaultJDBCAdapter
Date Thu, 07 Jan 2010 21:18:07 GMT

I was performing an endurance test on ActiveMQ 5.3.0.  My test consisted of a
single queue with a number of producers and consumers.  The producers send
very simple text messages and the consumers simply receive the message and
do no other processing.  Using the OOTB configuration (which uses -Xmx512m)
with JDBC persistence to an Oracle database, I found the system came to a
halt at around 10.3 million messages.  Looking at the VM revealed a very
full heap and tiny gains from garbage collection.  Restarting the broker
allows the system to run again.

To determine the cause of the exhausted heap, I took a series of heap dumps
over time.  My examination of the heaps showed that the number of live
instances of TreeMap$Entry and Long were increasing linearly with the number
of messages.  The Longs were owned by the TreeMap$Entry objects.  The
TreeMap$Entry objects could be tracked back to the TreeSet<Long> instance
from the lastRecoveredMessageIds field in DefaultJDBCAdapter.

The only method that uses lastRecoveredMessageIds is:
public void doRecoverNextMessages(TransactionContext c, ActiveMQDestination
destination, long nextSeq,
            int maxReturned, JDBCMessageRecoveryListener listener) throws
Exception

As the listener is called to recover a message, the id is added to this set. 
The id is only removed from this set if it is encountered on future run of
doRecoverNextMessages when it is added to the cleanupIds list.  The SQL that
is executed at the beginning of the method filters messages based on having
an id greater than nextSeq.  If nextSeq is always large enough, an id is
never added to cleanupIds and consequently never removed from
lastRecoveredMessageIds.

I saw that the use of lastRecoveredMessageIds was introduced with AMQ-1918. 
I hesitate to offer a solution, because I do not fully understand that
issue.  Also, AMQ-2436 synchronizes the TreeSet, but that should have no
effect on this issue.

Please let me know if I can supply any additional files or information to
help solve this issue.

Jim Harter

-- 
View this message in context: http://old.nabble.com/Memory-Leak-in-DefaultJDBCAdapter-tp27067181p27067181.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.


Mime
View raw message