lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kyrill Alyoshin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2237) allow TimeLimitedCollector timer thread to be shutdown
Date Tue, 01 Feb 2011 18:40:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989317#comment-12989317
] 

Kyrill Alyoshin commented on LUCENE-2237:
-----------------------------------------

The runFlag should also be declared volatile. This is an important bug.

> allow TimeLimitedCollector timer thread to be shutdown
> ------------------------------------------------------
>
>                 Key: LUCENE-2237
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2237
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.4, 2.4.1, 2.9, 2.9.1, 3.0
>            Reporter: Chris Darroch
>         Attachments: LUCENE-2237-2_4.patch
>
>
> When using Tomcat 6 and Solr 1.3 (with Lucene 2.4) we found that if we caused Tomcat
to reload our .war files a number of times, we would eventually see PermGen memory errors
where the JVM' s GC reported that all "permanent generation" memory had been consumed and
none could be freed.  This turns out to be a fairly common issue when using Tomcat's autoDeploy
feature (or similar features of other application servers).  See, for example:
> http://ampedandwired.dreamhosters.com/2008/05/09/causes-of-java-permgen-memory-leaks/
> http://cornelcreanga.com/2009/02/how-to-prevent-memory-leaks-when-reloading-web-applications/
> http://www.samaxes.com/2007/10/classloader-leaks-and-permgen-space/
> http://blogs.sun.com/fkieviet/entry/how_to_fix_the_dreaded
> My understanding of the issue is that when reloading a webapp, Tomcat starts by releasing
all of its references to the ClassLoader used to load the previous version of the application.
 Then it creates a new ClassLoader which reloads the application.  The old ClassLoader and
old version of the app are left to the garbage collector to be cleaned up.  However, if the
app itself hold references to the ClassLoader, the GC may not be able to ascertain that the
ClassLoader is truly unused, in which case, it and the entire old version of app remain in
memory.  If one causes a sufficient number of app reloads, eventually PermGen space is exhausted.
> The particular issue we had with Solr and Lucene was that Lucene's TimeLimitedCollector
creates a thread which is not shut down anywhere; this in turn seems to prevent Tomcat from
unloading the ClassLoader.  To solve this I applied a minor patch to TimeLimitedCollector
which adds a flag variable controlling the timer thread's loop and some methods to set it
so the thread will exit.
> The stopThread() method can then be called by an application such as Solr from a class
registered as a servlet context listener; when the server is unloading the application the
listener will execute and in turn stop the timer thread.  My testing during multiple reloads
of solr.war with and without these patches indicates that without them, we consistently get
PermGen errors, and with them, once the PermGen is nearly exhausted (which may take a lot
of reloads, e.g., 10-15!), the GC is able to free space and no PermGen errors occur.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message