lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Sturlese <marc.sturl...@gmail.com>
Subject Re: memory leak with CustomComparatorSource class variables
Date Sat, 13 Jun 2009 13:30:09 GMT

Thanks Mike, really useful info. I have dowloaded the latest Lucene 2.9-dev
to test the implementation of a FieldComparatorSource but the API
documentation doesn't seem to be availabe. 

I can access to the class MissingStringLastComparatorSource:
http://lucene.apache.org/solr/api/org/apache/solr/search/MissingStringLastComparatorSource.html
>From there I try to link to org.apache.lucene.search.FieldComparatorSource
but get a 404 error.
Any idea how can I get access to that documentation?

Thanks in advance!


Michael McCandless-2 wrote:
> 
> On Fri, Jun 12, 2009 at 6:09 PM, Marc Sturlese<marc.sturlese@gmail.com>
> wrote:
> 
>> I have noticed I am experiencing sort of a memory leak with a
>> CustomComparatorSource (wich implements SortComparatorSource).
>> I have a HashMap declared as variable of class in CustomComparatorSource:
> 
> This is unfortunately a known and rather horrific trap, in Lucene.
> 
> Lucene's field sorting implementation (FieldSortedHitQueue) caches the
> comparators use during sorting (in its static package private
> Comparators field).  They are weakly keyed by IndexReader, so that
> when the IndexReader is closed, the cache entries are cleared.  When
> sorting by field this is normally OK since we hold a single entry for
> that field.
> 
> But when you provide a SortComparatorSource, it's included in the
> cache key.  So, if you don't implement hashCode/equals (correctly), or
> using a singleton or restricted set of instances (say), then suddenly
> every new instance of your SortComparatorSource will enter the cache
> and not be cleared until you close that reader.  It easily results in
> a catastrophic, extremely unexpected memory leak.
> 
> Lucene 2.9 has fixed this, as a side effect of the move to per-segment
> field sorting.  SortComparatorSource is replaced by
> FieldComparatorSource, and this caching of comparators is no longer
> done.  Another reason to get 2.9 out sooner rather than later!
> 
> Mike
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/memory-leak-with-CustomComparatorSource-class-variables-tp24006806p24012496.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message