tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <>
Subject Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Date Tue, 07 May 2013 22:06:01 GMT
Hash: SHA256


On 5/7/13 4:33 PM, Michael-O wrote:
> Am 2013-05-07 17:20, schrieb Christopher Schultz:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>> Michael,
>> On 5/7/13 6:43 AM, Michael-O wrote:
>>>> Von: "Mark Thomas" <> On 07/05/2013 10:25, 
>>>> Michael-O wrote:
>>>>>> Von: "Mark Thomas" <> On 07/05/2013
>>>>>> 09:16, Michael-O wrote:
>>>>>>> Hi folks,
>>>>>>> I recently enabled a QueryTimeoutInterceptor with 
>>>>>>> queryTimeout of 60 seconds in a JDBC Pool data source 
>>>>>>> (7.0.37). When the app was shut down, Tomcat said: "The
>>>>>>> web application [/...] appears to have started a thread
>>>>>>> named [OracleTimeoutPollingThread] but has failed to
>>>>>>> stop it..."
>>>>>>> We are using Oracle 11.2g with JDBC drivers.
>>>>>>> I have figured out that this thread is spawned by the
>>>>>>> driver itself. According to this Stackoverflow answer
>>>>>>> [1] this is a long-living thread, same says the JDBC
>>>>>>> FAQ [2] of Oracle.
>>>>>>> The thread seems to work like Pool's PoolCleaner
>>>>>>> thread. A few month a ago I reported the same issue
>>>>>>> with the PoolCleaner thread and Filip fixed the class
>>>>>>> loader orders.
>>>>>>> Can this be a false-positive by the memory leak
>>>>>>> detector since this thread lives only once in the
>>>>>>> entire VM?
>>>>>> No. It is a memory leak and either or bug in your
>>>>>> application or in the JDBC driver.
>>>>>> Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib
>>>>>> or WEB-INF/lib
>>>>> The driver is in the $CATALINA_HOME/lib only where 
>>>>> $CATALINA_BASE != $CATALINA_HOME. This was done for a
>>>>> single webapp for testing purposes.
>>>>> Does this make a difference?
>>>> The important thing is that it isn't in WEB-INF/lib.
>>>>> How do you know that this is not a false-positive?
>>>> Experience, a lot of research into memory leaks and I wrote 
>>>> Tomcat's memory leak detection code.
>>>>> If you really know for sure, I can open a service request
>>>>> with Oracle Support.
>>>> Good luck with that.
>>>> The problem is that when the driver creates the thread it
>>>> does so when the current class loader is the web application
>>>> class loader. That means that the Thread will be created with
>>>> a context class loader set to the web application class
>>>> loader. That will trigger a memory leak when the web
>>>> application is stopped because a reference is retained to the
>>>> web application's class loader.
>>>> What the driver should do is, after it creates the thread,
>>>> set the thread's context class loader to the class loader
>>>> that loaded the driver.
>>>> What you are seeing is a variation of the leak described on
>>>> page 15 of [1].
>>> After reading the slides and your explanation this makes
>>> sense. It's the same issue as "Pool cleaner thread should be
>>> created using the classloader that loaded the pool, not the
>>> context loader (fhanik)" fixed in 7.0.27.
>>> I will file SR and let you know.
>> Note that you might be able to write your own code to mitigate
>> this problem, depending on exactly when that thread is created.
>> If the timeout thread isn't created until you actually try to
>> issue a query with a timeout, try something like this in a
>> ServletContextListener's contextInitialized method:
>> // NOTE: No resource management is being done in this example
>> // Get the current ClassLoader -- should be WebappClassLoader 
>> ClassLoader cl = Thread.currentThread().getContextClassLoader();
>> // WebappClassLoader.getParent should be "common" loader 
>> Thread.currentThread().setContextClassLoader(cl.getParent());
>> try { Connection conn = ...; // However you get a connection 
>> Statement s = conn.createStatement(); s.setQueryTimeout(5000); //
>> Doesn't really matter what TO is s.execute("SELECT 1 FROM dual",
>> ); } .... finally { // Pop back to the original ClassLoader 
>> Thread.currentThread().setContextClassLoader(cl); }
>> This is the exact technique that Tomcat's 
>> JreMemoryLeakPreventionListener uses to prevent ClassLoader
>> leaks.
> Yes, this looks like the way JDBC Pool does with the Pool Cleaner.
> I would go for this as a last resort.
>> A couple of notes:
>> 1. This won't work under a SecurityManager. If you need to
>> operate under a SecurityManager, have a look at the 
>> JreMemoryLeakPreventionListener code and adapt it to the above.
>> 2. If the Oracle driver launches the thread when the DataSource
>> is created, it might happen too early for a
>> ServletContextListener to intervene. In that case, simply modify
>> the JreMEmoryLeakPreventionListener code directly. Patches are
>> always welcome.
> Not, it is not. I have attached VisualVM to Tomcat VM and have seen
> that the thread is forked when the first query is executed.

That's good news: it means that if you need it fixed *right away*, you
don't need to hack-and-recompile Tomcat: you can just do it in your
own ServletContextListener.

>> 3. If Oracle fixes this bug, Tomcat should not prevent against
>> it. If Oracle refuses to acknowledge/fix/etc. this bug, then it
>> may make sense to include such code in
>> JreMemoryLeakPreventionListener, but it should probably be done
>> in such a way that a) it's not enabled by default and b) the
>> query used for triggering the Thread to be created is
>> user-selectable with maybe a reasonable default (like "SELECT 1 
>> FROM dual", as that tends to be valid in most RDBMSs).
> I am already in contact with an Oracle engineer who has received a
> demo WAR file to reproduce this issue. If Oracle won't, maybe some
> generic approach would be advisable but in the in 
> JreMemoryLeakPreventionListener but rather in Tomcat JDBC Pool

I think this might be a rather esoteric thing to exercise, but one
could certainly add it as an interceptor that is not enabled by default.

> I do not know how other RDBMS vendors implement the timeout
> function.

MySQL has taken some time to fix similar/related bugs, but is
(eventually) willing to fix them:

- -chris
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools -
Comment: Using GnuPG with Thunderbird -


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message