lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Lu (JIRA)" <>
Subject [jira] Commented: (LUCENE-1383) Work around ThreadLocal's "leak"
Date Sat, 13 Sep 2008 15:48:44 GMT


Chris Lu commented on LUCENE-1383:

I can confirm this patch fixed the memory problem, in general. Thanks!

However, if not nitpicking, I did noticed in this patch, during the transition time between
the previous index searcher/reader close and the new index searcher/reader open, there is
a bump in the memory graph.
There are 2 RAMDirectory instances in the memory for a short period of time.

In previous Lucene version without Lucene-1195, the memory graph looks more clean, with a
V dip.
____  ____

And in one index refresh during the same index run, there were 2 RAMDirectory in the memory
for a fairly long time. But not repeatable.

So I suspect the closing is kind of delayed. And sometimes it could be missed.

Attaching the screenshot for the memory bump, and 2 the memory graphs of the begin and after
of 2RAMDirecory in the memory(needed 2 because the time is fairly long)

> Work around ThreadLocal's "leak"
> --------------------------------
>                 Key: LUCENE-1383
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 1.9, 2.0.0, 2.1, 2.2, 2.3, 2.3.1, 2.3.2
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 2.4
>         Attachments: LUCENE-1383.patch, ScreenHunter_01 Sep. 13 08.40.jpg
> Java's ThreadLocal is dangerous to use because it is able to take a
> surprisingly very long time to release references to the values you
> store in it.  Even when a ThreadLocal instance itself is GC'd, hard
> references to the values you had stored in it are easily kept for
> quite some time later.
> While this is not technically a "memory leak", because eventually
> (when the underlying Map that stores the values cleans up its "stale"
> references) the hard reference will be cleared, and GC can proceed,
> its end behavior is not different from a memory leak in that under the
> right situation you can easily tie up far more memory than you'd
> expect, and then hit unexpected OOM error despite allocating an
> extremely large heap to your JVM.
> Lucene users have hit this many times.  Here's the most recent thread:
> And here's another:
> And then there's LUCENE-436 and LUCENE-529 at least.
> A google search for "ThreadLocal leak" yields many compelling hits.
> Sun does this for performance reasons, but I think it's a terrible
> trap and we should work around it with Lucene.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message