lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aleksey Serba" <>
Subject Re: Custom sorting - memory leaks
Date Mon, 21 Aug 2006 11:15:39 GMT
Hi Chris,

On 5/17/06, Peter Keegan <peterlkeegan[at]> wrote:
> Suppose I have a custom sorting 'DocScoreComparator' for computing distances
> on each search hit from a specified coordinate (similar to the
> DistanceComparatorSource example in LIA). Assume that the 'specified
> coordinate' is different for each query. This means a new custom comparator
> must be created for each query, which is ok. However, Lucene caches the
> comparator even though it will never be reused. This could result in heavy
> memory usage if many queries are performed before the IndexReader is
> updated.

Lets suppose another custom sortings - random, using dynamically
changed values from database, etc. Everytime lucene caches _useless_
custom comparators and application developer has no any control on it.

Also, I have no evidence to close IndexReader if my index is static.


On 8/18/06, Chris Hostetter <> wrote:
> : You can reproduce OutOfMemory easily. I've attach test files - this is
> : altered DistanceSortingTest example from LIA book. Also you can
> : profile it and see caching of distances arrays.
> An OutOfMemory error is differnet from a memory leak.  Sorting with a
> custom Comparator does in fact use a lot of memory -- and if your Heap
> size is not big enough you may get an OutOfMemory -- but that doesn't mean
> you have a memory leak.  A Memory leak is a problem where memory is
> allocated for objects, but not freed up once those objects are no longer
> of use -- just becuase you aren't using those objects, doesn't mean they
> aren't "of use" and it doesn't mean they won't be freed up once some
> futher action is taken (ie: closing of an IndexReader)
> Are you sure you've found an actual memory leak? ... ie: do you have a
> demonstratable test case that shows heap usage growing without bound over
> time?   If so then seeing *that* code would be helpful to figure out if
> there is in fact a memory leak in Lucene, or if (perhaps) there is a
> mistaken assumption in your code (ie; not closing an IndexReader)
> -Hoss
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message