lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Lu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1383) Work around ThreadLocal's "leak"
Date Sat, 13 Sep 2008 15:48:44 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630784#action_12630784
] 

Chris Lu commented on LUCENE-1383:
----------------------------------

I can confirm this patch fixed the memory problem, in general. Thanks!

However, if not nitpicking, I did noticed in this patch, during the transition time between
the previous index searcher/reader close and the new index searcher/reader open, there is
a bump in the memory graph.
  ____/\____
There are 2 RAMDirectory instances in the memory for a short period of time.

In previous Lucene version without Lucene-1195, the memory graph looks more clean, with a
V dip.
____  ____
         \/

And in one index refresh during the same index run, there were 2 RAMDirectory in the memory
for a fairly long time. But not repeatable.

So I suspect the closing is kind of delayed. And sometimes it could be missed.

Attaching the screenshot for the memory bump, and 2 the memory graphs of the begin and after
of 2RAMDirecory in the memory(needed 2 because the time is fairly long)

> Work around ThreadLocal's "leak"
> --------------------------------
>
>                 Key: LUCENE-1383
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1383
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 1.9, 2.0.0, 2.1, 2.2, 2.3, 2.3.1, 2.3.2
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 2.4
>
>         Attachments: LUCENE-1383.patch, ScreenHunter_01 Sep. 13 08.40.jpg
>
>
> Java's ThreadLocal is dangerous to use because it is able to take a
> surprisingly very long time to release references to the values you
> store in it.  Even when a ThreadLocal instance itself is GC'd, hard
> references to the values you had stored in it are easily kept for
> quite some time later.
> While this is not technically a "memory leak", because eventually
> (when the underlying Map that stores the values cleans up its "stale"
> references) the hard reference will be cleared, and GC can proceed,
> its end behavior is not different from a memory leak in that under the
> right situation you can easily tie up far more memory than you'd
> expect, and then hit unexpected OOM error despite allocating an
> extremely large heap to your JVM.
> Lucene users have hit this many times.  Here's the most recent thread:
>   http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200809.mbox/%3C6e3ae6310809091157j7a9fe46bxcc31f6e63305fcdc%40mail.gmail.com%3E
> And here's another:
>   http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200807.mbox/%3CF5FC94B2-E5C7-40C0-8B73-E12245B91CEE%40mikemccandless.com%3E
> And then there's LUCENE-436 and LUCENE-529 at least.
> A google search for "ThreadLocal leak" yields many compelling hits.
> Sun does this for performance reasons, but I think it's a terrible
> trap and we should work around it with Lucene.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message