lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: memory leak with CustomComparatorSource class variables
Date Sat, 13 Jun 2009 10:50:35 GMT
On Fri, Jun 12, 2009 at 6:09 PM, Marc Sturlese<marc.sturlese@gmail.com> wrote:

> I have noticed I am experiencing sort of a memory leak with a
> CustomComparatorSource (wich implements SortComparatorSource).
> I have a HashMap declared as variable of class in CustomComparatorSource:

This is unfortunately a known and rather horrific trap, in Lucene.

Lucene's field sorting implementation (FieldSortedHitQueue) caches the
comparators use during sorting (in its static package private
Comparators field).  They are weakly keyed by IndexReader, so that
when the IndexReader is closed, the cache entries are cleared.  When
sorting by field this is normally OK since we hold a single entry for
that field.

But when you provide a SortComparatorSource, it's included in the
cache key.  So, if you don't implement hashCode/equals (correctly), or
using a singleton or restricted set of instances (say), then suddenly
every new instance of your SortComparatorSource will enter the cache
and not be cleared until you close that reader.  It easily results in
a catastrophic, extremely unexpected memory leak.

Lucene 2.9 has fixed this, as a side effect of the move to per-segment
field sorting.  SortComparatorSource is replaced by
FieldComparatorSource, and this caching of comparators is no longer
done.  Another reason to get 2.9 out sooner rather than later!

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message