lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <>
Subject [jira] Commented: (LUCENE-1593) Optimizations to TopScoreDocCollector and TopFieldCollector
Date Wed, 29 Apr 2009 18:27:31 GMT


Shai Erera commented on LUCENE-1593:

bq, I'm not sure we can make such a change even in 3.0. Ie, all that's "special" about 3.0
is we get to remove deprecated APIs, and begin using Java 1.5 language features.

I'd like to discuss that in a separate thread, where it will have more visibility ... I'm
a bit puzzled by what 3.0 means, but it should be discussed outside the scope of this issue.

bq. So maybe we can make a new abstract class called AbstractWeight ...

I think we should have an issue handling interfaces deprecation in general for 2.9, since
just deprecating Weight does not solve it. You'd have to deprecate* methods
which accept Weight, but Searchable is an interface, so you might want to deprecate it entirely
and create an AbstractSearchable? That I think also deserves its own thread, don't you think?

When I thought about the ambiguity that we have in BS2 between score(Collector) and next()/skipTo()
and the proposal to have topScorer() and scorer(), I thought that perhaps we can make the
following change (we'd have to solve the Weight-interface problem first):
# Define on Weight a score(IndexReader, Collector) API which will be called instead of the
topScorer() proposal.
# Keep the scorer(IndexReader) API - this should be used for iterating over the Scorer.
# Make Scorer.score(Collector) package-private so that it can still be used by Weight.score(IndexReader,
Collector), but not by anyone else. That will effectively remove that API from Scorer, but
still keep the impl there so we make the least amount of changes to the current Scorers and
#* We should document that it should not be used, even from inside Lucene's code unless there's
a really good reason. Everyone, including Lucene should use the Weight.score(IndexReader,
Collector) API.

That should present a clean and clear API, i.e. topScorer() and scorer() might not be understood
well, and we'd need to document their usage clearly, and we don't have a way to enforce that
once topScorer() is called, score(Collector) will be the only method that's used and not next()/skipTo().

> Optimizations to TopScoreDocCollector and TopFieldCollector
> -----------------------------------------------------------
>                 Key: LUCENE-1593
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Shai Erera
>             Fix For: 2.9
>         Attachments: LUCENE-1593.patch,
> This is a spin-off of LUCENE-1575 and proposes to optimize TSDC and TFC code to remove
unnecessary checks. The plan is:
> # Ensure that IndexSearcher returns segements in increasing doc Id order, instead of
> # Change TSDC and TFC's code to not use the doc id as a tie breaker. New docs will always
have larger ids and therefore cannot compete.
> # Pre-populate HitQueue with sentinel values in TSDC (score = Float.NEG_INF) and remove
the check if reusableSD == null.
> # Also move to use "changing top" and then call adjustTop(), in case we update the queue.
> # some methods in Sort explicitly add SortField.FIELD_DOC as a "tie breaker" for the
last SortField. But, doing so should not be necessary (since we already break ties by docID),
and is in fact less efficient (once the above optimization is in).
> # Investigate PQ - can we deprecate insert() and have only insertWithOverflow()? Add
a addDummyObjects method which will populate the queue without "arranging" it, just store
the objects in the array (this can be used to pre-populate sentinel values)?
> I will post a patch as well as some perf measurements as soon as I have them.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message