lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: Optimization of memory usage in PriorityQueue
Date Mon, 22 Jun 2009 13:40:24 GMT
>
> Though, are Lucene's core collectors reusable?  If you did really want
> say 100K results out of each search (very unusual), it'd be nice to
> not have to throw away the Collector/PQ each time.
>

PQ has clear(), but it does not really allow you to reuse it, it just
removes all the elements. So if we want to reuse a PQ, then we need to add a
reset() (with a default impl of calling clear()) to both TSDC and PQ. TSDC
will delegate the call to PQ, which is actually HitQueue, which will iterate
on all the elements and reset them to sentinel values.

TopFieldCollector's reset() will do the same, delegating the call to its own
PQ (FieldValueHitQueue), which will do nothing (call clear()).

Do you think it's worth it? If so, should we also add reset() to Collector?

Shai

On Mon, Jun 22, 2009 at 12:29 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> On Mon, Jun 22, 2009 at 3:25 AM, Shai Erera<serera@gmail.com> wrote:
>
> > Or ... we can do nothing, and say that one can write his own Collector,
> and
> > use Sun's PQ (or any other), if one has a case like "I need 10K results,
> but
> > I don't know how many are there, and specifically I want to optimize for
> the
> > case of 1 result".
>
> +1
>
> I think Lucene's current PQ is optimized for the [very] common case,
> and if someone would like to eg swap to Sun's PQ impl, the custom
> Collector API is the best route.  (And, I'd love to hear back on how
> the performance compares!  If Sun has a faster PQ than Lucene, we
> should fix that ;) ).
>
> Though, are Lucene's core collectors reusable?  If you did really want
> say 100K results out of each search (very unusual), it'd be nice to
> not have to throw away the Collector/PQ each time.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
View raw message