lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Sturlese <>
Subject Re: memory leak with CustomComparatorSource class variables
Date Sat, 13 Jun 2009 13:30:09 GMT

Thanks Mike, really useful info. I have dowloaded the latest Lucene 2.9-dev
to test the implementation of a FieldComparatorSource but the API
documentation doesn't seem to be availabe. 

I can access to the class MissingStringLastComparatorSource:
>From there I try to link to
but get a 404 error.
Any idea how can I get access to that documentation?

Thanks in advance!

Michael McCandless-2 wrote:
> On Fri, Jun 12, 2009 at 6:09 PM, Marc Sturlese<>
> wrote:
>> I have noticed I am experiencing sort of a memory leak with a
>> CustomComparatorSource (wich implements SortComparatorSource).
>> I have a HashMap declared as variable of class in CustomComparatorSource:
> This is unfortunately a known and rather horrific trap, in Lucene.
> Lucene's field sorting implementation (FieldSortedHitQueue) caches the
> comparators use during sorting (in its static package private
> Comparators field).  They are weakly keyed by IndexReader, so that
> when the IndexReader is closed, the cache entries are cleared.  When
> sorting by field this is normally OK since we hold a single entry for
> that field.
> But when you provide a SortComparatorSource, it's included in the
> cache key.  So, if you don't implement hashCode/equals (correctly), or
> using a singleton or restricted set of instances (say), then suddenly
> every new instance of your SortComparatorSource will enter the cache
> and not be cleared until you close that reader.  It easily results in
> a catastrophic, extremely unexpected memory leak.
> Lucene 2.9 has fixed this, as a side effect of the move to per-segment
> field sorting.  SortComparatorSource is replaced by
> FieldComparatorSource, and this caching of comparators is no longer
> done.  Another reason to get 2.9 out sooner rather than later!
> Mike
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View this message in context:
Sent from the Lucene - Java Users mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message