activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dejan Bosanac (JIRA)" <>
Subject [jira] Assigned: (AMQ-2563) Memory Leak in DefaultJDBCAdapter
Date Tue, 02 Feb 2010 09:39:13 GMT


Dejan Bosanac reassigned AMQ-2563:

    Assignee: Dejan Bosanac

> Memory Leak in DefaultJDBCAdapter
> ---------------------------------
>                 Key: AMQ-2563
>                 URL:
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.3.0
>         Environment: Java 1.6.0_07-b06 on Solaris (5.10) sparcv9
>            Reporter: Jim Harter
>            Assignee: Dejan Bosanac
>             Fix For: 5.3.1, 5.4.0
>         Attachments: AMQTest.war,, jdbc-test.xml
> I was performing an endurance test on ActiveMQ 5.3.0.  My test consisted of a single
queue with a number of producers and consumers.  The producers send very simple text messages
and the consumers simply receive the message and do no other processing.  Using the OOTB configuration
(which uses -Xmx512m) with JDBC persistence to an Oracle database, I found the system came
to a halt at around 10.3 million messages.  Looking at the VM revealed a very full heap and
tiny gains from garbage collection.  Restarting the broker allows the system to run again.
> To determine the cause of the exhausted heap, I took a series of heap dumps over time.
 My examination of the heaps showed that the number of live instances of TreeMap$Entry and
Long were increasing linearly with the number of messages.  The Longs were owned by the TreeMap$Entry
objects.  The TreeMap$Entry objects could be tracked back to the TreeSet<Long> instance
from the lastRecoveredMessageIds field in DefaultJDBCAdapter.
> The only method that uses lastRecoveredMessageIds is:
> public void doRecoverNextMessages(TransactionContext c, ActiveMQDestination destination,
long nextSeq,
>             int maxReturned, JDBCMessageRecoveryListener listener) throws Exception
> As the listener is called to recover a message, the id is added to this set.  The id
is only removed from this set if it is encountered on future run of doRecoverNextMessages
when it is added to the cleanupIds list.  The SQL that is executed at the beginning of the
method filters messages based on having an id greater than nextSeq.  If nextSeq is always
large enough, an id is never added to cleanupIds and consequently never removed from lastRecoveredMessageIds.
> I saw that the use of lastRecoveredMessageIds was introduced with AMQ-1918.  Also, AMQ-2436
synchronizes the TreeSet, but that should have no effect on this issue.
> Dejan mentioned some work done on JDBC persistence and a memory leak fix in association
with AMQ-2519.  I have performed the same test on the latest snapshots of 5.3.1 and 5.4. 
The same results were observed on those as well.  Also, work done for AMQ-2519 modified JDBCMessageStore,
JDBCTopicMessageStore, and JDBCPersistenceAdapter.  I believe the issue is in DefaultJDBCAdapter.
> To reproduce:
> - Start a broker using JDBC persistence (sample configuration file will be attached)
> - Start a number of producers and consumers using the same queue on that broker (sample
WAR file will be attached.  Modify WEB-INF/applicationContext.xml to set the queue name and
broker URL.  The war file currently uses TestQueue and tcp://localhost:61616, respectively.)
> - Wait for about 20,000 messages to be processed (you can use less; it just gets easier
in the heap dumps to see after about 20,000 messages.  29 bytes are retained per message that
is processed)
> - Look at the old generation size after a full garbage collection over time.  It grows
> - Obtain a heap dump.  The heap dump will show a number of retained instances of TreeMap$Entry
and Long.  (sample heap dump will be attached)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message