lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul നോബിള്‍ नोब्ळ्" <noble.p...@gmail.com>
Subject Re: ThreadLocal causing memory leak with J2EE applications
Date Thu, 11 Sep 2008 04:30:15 GMT
When I look at the reference tree That is the feeling I get. if you
held a WeakReference it would get released .
 |- base of org.apache.lucene.index.CompoundFileReader$CSIndexInput
              |- input of org.apache.lucene.index.SegmentTermEnum
                  |- value of java.lang.ThreadLocal$ThreadLocalMap$Entry

On Wed, Sep 10, 2008 at 8:39 PM, Chris Lu <chris.lu@gmail.com> wrote:
> Does this make any difference?
> If I intentionally close the searcher and reader failed to release the
> memory, I can not rely on some magic of JVM to release it.
> --
> Chris Lu
> -------------------------
> Instant Scalable Full-Text Search On Any Database/Application
> site: http://www.dbsight.net
> demo: http://search.dbsight.com
> Lucene Database Search in 3 minutes:
> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
> DBSight customer, a shopping comparison site, (anonymous per request) got
> 2.6 Million Euro funding!
>
> On Wed, Sep 10, 2008 at 4:03 AM, Noble Paul നോബിള്‍ नोब्ळ्
> <noble.paul@gmail.com> wrote:
>>
>> Why do you need to keep a strong reference?
>> Why not a WeakReference ?
>>
>> --Noble
>>
>> On Wed, Sep 10, 2008 at 12:27 AM, Chris Lu <chris.lu@gmail.com> wrote:
>> > The problem should be similar to what's talked about on this discussion.
>> > http://lucene.markmail.org/message/keosgz2c2yjc7qre?q=ThreadLocal
>> >
>> > There is a memory leak for Lucene search from Lucene-1195.(svn r659602,
>> > May23,2008)
>> >
>> > This patch brings in a ThreadLocal cache to TermInfosReader.
>> >
>> > It's usually recommended to keep the reader open, and reuse it when
>> > possible. In a common J2EE application, the http requests are usually
>> > handled by different threads. But since the cache is ThreadLocal, the
>> > cache
>> > are not really usable by other threads. What's worse, the cache can not
>> > be
>> > cleared by another thread!
>> >
>> > This leak is not so obvious usually. But my case is using RAMDirectory,
>> > having several hundred megabytes. So one un-released resource is obvious
>> > to
>> > me.
>> >
>> > Here is the reference tree:
>> > org.apache.lucene.store.RAMDirectory
>> >  |- directory of org.apache.lucene.store.RAMFile
>> >      |- file of org.apache.lucene.store.RAMInputStream
>> >          |- base of
>> > org.apache.lucene.index.CompoundFileReader$CSIndexInput
>> >              |- input of org.apache.lucene.index.SegmentTermEnum
>> >                  |- value of java.lang.ThreadLocal$ThreadLocalMap$Entry
>> >
>> >
>> > After I switched back to svn revision 659601, right before this patch is
>> > checked in, the memory leak is gone.
>> > Although my case is RAMDirectory, I believe this will affect disk based
>> > index also.
>> >
>> > --
>> > Chris Lu
>> > -------------------------
>> > Instant Scalable Full-Text Search On Any Database/Application
>> > site: http://www.dbsight.net
>> > demo: http://search.dbsight.com
>> > Lucene Database Search in 3 minutes:
>> >
>> > http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>> > DBSight customer, a shopping comparison site, (anonymous per request)
>> > got
>> > 2.6 Million Euro funding!
>> >
>>
>>
>>
>> --
>> --Noble Paul
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
>



-- 
--Noble Paul
Mime
View raw message