tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael-O <1983-01...@gmx.net>
Subject Re: Possible false-postive with JreMemoryLeakPreventionListener and Tomcat's JDBC Pool
Date Tue, 07 May 2013 20:33:13 GMT
Hello Christopher,

Am 2013-05-07 17:20, schrieb Christopher Schultz:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Michael,
>
> On 5/7/13 6:43 AM, Michael-O wrote:
>>> Von: "Mark Thomas" <markt@apache.org> On 07/05/2013 10:25,
>>> Michael-O wrote:
>>>>
>>>>> Von: "Mark Thomas" <markt@apache.org> On 07/05/2013 09:16,
>>>>> Michael-O wrote:
>>>>>> Hi folks,
>>>>>>
>>>>>> I recently enabled a QueryTimeoutInterceptor with
>>>>>> queryTimeout of 60 seconds in a JDBC Pool data source
>>>>>> (7.0.37). When the app was shut down, Tomcat said: "The web
>>>>>> application [/...] appears to have started a thread named
>>>>>> [OracleTimeoutPollingThread] but has failed to stop it..."
>>>>>>
>>>>>> We are using Oracle 11.2g with 11.2.0.3 JDBC drivers. I
>>>>>> have figured out that this thread is spawned by the driver
>>>>>> itself. According to this Stackoverflow answer [1] this is
>>>>>> a long-living thread, same says the JDBC FAQ [2] of
>>>>>> Oracle.
>>>>>>
>>>>>> The thread seems to work like Pool's PoolCleaner thread. A
>>>>>> few month a ago I reported the same issue with the
>>>>>> PoolCleaner thread and Filip fixed the class loader
>>>>>> orders.
>>>>>>
>>>>>> Can this be a false-positive by the memory leak detector
>>>>>> since this thread lives only once in the entire VM?
>>>>>
>>>>> No. It is a memory leak and either or bug in your application
>>>>> or in the JDBC driver.
>>>>>
>>>>> Where is the Oracle JDBC driver? CATALINA_[BASE|HOME]/lib or
>>>>> WEB-INF/lib
>>>>
>>>> The driver is in the $CATALINA_HOME/lib only where
>>>> $CATALINA_BASE != $CATALINA_HOME. This was done for a single
>>>> webapp for testing purposes.
>>>>
>>>> Does this make a difference?
>>>
>>> The important thing is that it isn't in WEB-INF/lib.
>>>
>>>> How do you know that this is not a false-positive?
>>>
>>> Experience, a lot of research into memory leaks and I wrote
>>> Tomcat's memory leak detection code.
>>>
>>>> If you really know for sure, I can open a service request with
>>>> Oracle Support.
>>>
>>> Good luck with that.
>>>
>>> The problem is that when the driver creates the thread it does so
>>> when the current class loader is the web application class
>>> loader. That means that the Thread will be created with a context
>>> class loader set to the web application class loader. That will
>>> trigger a memory leak when the web application is stopped because
>>> a reference is retained to the web application's class loader.
>>>
>>> What the driver should do is, after it creates the thread, set
>>> the thread's context class loader to the class loader that loaded
>>> the driver.
>>>
>>> What you are seeing is a variation of the leak described on page
>>> 15 of [1].
>>
>> After reading the slides and your explanation this makes sense.
>> It's the same issue as "Pool cleaner thread should be created using
>> the classloader that loaded the pool, not the context loader
>> (fhanik)" fixed in 7.0.27.
>>
>> I will file SR and let you know.
>
> Note that you might be able to write your own code to mitigate this
> problem, depending on exactly when that thread is created. If the
> timeout thread isn't created until you actually try to issue a query
> with a timeout, try something like this in a ServletContextListener's
> contextInitialized method:
>
>      // NOTE: No resource management is being done in this example
>
>      // Get the current ClassLoader -- should be WebappClassLoader
>      ClassLoader cl = Thread.currentThread().getContextClassLoader();
>
>      // WebappClassLoader.getParent should be "common" loader
>      Thread.currentThread().setContextClassLoader(cl.getParent());
>
>      try
>      {
>        Connection conn = ...; // However you get a connection
>        Statement s = conn.createStatement();
>        s.setQueryTimeout(5000); // Doesn't really matter what TO is
>        s.execute("SELECT 1 FROM dual", );
>      }
>      ....
>      finally
>      {
>        // Pop back to the original ClassLoader
>        Thread.currentThread().setContextClassLoader(cl);
>      }
>
> This is the exact technique that Tomcat's
> JreMemoryLeakPreventionListener uses to prevent ClassLoader leaks.

Yes, this looks like the way JDBC Pool does with the Pool Cleaner. I 
would go for this as a last resort.

> A
> couple of notes:
>
> 1. This won't work under a SecurityManager. If you need to operate
> under a SecurityManager, have a look at the
> JreMemoryLeakPreventionListener code and adapt it to the above.
>
> 2. If the Oracle driver launches the thread when the DataSource is
> created, it might happen too early for a ServletContextListener to
> intervene. In that case, simply modify the
> JreMEmoryLeakPreventionListener code directly. Patches are always welcome.

Not, it is not. I have attached VisualVM to Tomcat VM and have seen that 
the thread is forked when the first query is executed.

> 3. If Oracle fixes this bug, Tomcat should not prevent against it. If
> Oracle refuses to acknowledge/fix/etc. this bug, then it may make
> sense to include such code in JreMemoryLeakPreventionListener, but it
> should probably be done in such a way that a) it's not enabled by
> default and b) the query used for triggering the Thread to be created
> is user-selectable with maybe a reasonable default (like "SELECT 1
> FROM dual", as that tends to be valid in most RDBMSs).

I am already in contact with an Oracle engineer who has received a demo 
WAR file to reproduce this issue. If Oracle won't, maybe some generic 
approach would be advisable but in the in 
JreMemoryLeakPreventionListener but rather in Tomcat JDBC Pool IMHO.

I do not know how other RDBMS vendors implement the timeout function.

Michael


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message