lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Smith (JIRA)" <>
Subject [jira] Commented: (LUCENE-1821) Weight.scorer() not passed doc offset for "sub reader"
Date Fri, 21 Aug 2009 14:00:15 GMT


Tim Smith commented on LUCENE-1821:

It was an implementation detail. If you look at MultiSearcher, Searchable, Searcher and how
the API is put together, you can see we don't support that type of thing. I think its fairly
clear after a little thought.

You can limit your API's to handle just IndexSearchers, but as a project, we cannot.
I totally understand your resistance here.  I get that i'm really utilizing advanced lucene
concepts at very low levels (and these are subject to some changes that i will have to absorb
with new versions)

bq. Its okay to pass the Reader because its a contextless Reader. There is no value in also
passing a contextless Searcher
well, when you pass the Searcher that contains Reader, the Reader is no longer contextless.
also, the context of the Searcher can be fairly well defined (its a "leaf" Searcher. the one
that actually called Weight.scorer())

Also, looking a bit more at MultiSearcher semantics, sorting requires this "leaf" Searcher
context in order to work already
MultiSearcher just takes the top docs from each underlaying Searchable, adjusts the docids
to the MultiSearcher Context, and sends them through another priority queue
So, this "leaf" Searcher context concept is required by sorting already. 
I just want my Scorer to be given this "leaf" context as well

Also, since it is a "leaf" context, the Weight.scorer() method could have the following interface:
 * @param searcher The IndexSearcher that contains reader.
public Scorer scorer(IndexSearcher searcher, IndexReader reader, boolean allowDocsInOrder,
boolean topScorer);

then, with the patch i posted, i could call:
and i'm all set

> Weight.scorer() not passed doc offset for "sub reader"
> ------------------------------------------------------
>                 Key: LUCENE-1821
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Tim Smith
>             Fix For: 2.9
>         Attachments: LUCENE-1821.patch
> Now that searching is done on a per segment basis, there is no way for a Scorer to know
the "actual" doc id for the document's it matches (only the relative doc offset into the segment)
> If using caches in your scorer that are based on the "entire" index (all segments), there
is now no way to index into them properly from inside a Scorer because the scorer is not passed
the needed offset to calculate the "real" docid
> suggest having Weight.scorer() method also take a integer for the doc offset
> Abstract Weight class should have a constructor that takes this offset as well as a method
to get the offset
> All Weights that have "sub" weights must pass this offset down to created "sub" weights
> Details on workaround:
> In order to work around this, you must do the following:
> * Subclass IndexSearcher
> * Add "int getIndexReaderBase(IndexReader)" method to your subclass
> * during Weight creation, the Weight must hold onto a reference to the passed in Searcher
(casted to your sub class)
> * during Scorer creation, the Scorer must be passed the result of YourSearcher.getIndexReaderBase(reader)
> * Scorer can now rebase any collected docids using this offset
> Example implementation of getIndexReaderBase():
> {code}
> // NOTE: more efficient implementation can be done if you cache the result if gatherSubReaders
in your constructor
> public int getIndexReaderBase(IndexReader reader) {
>   if (reader == getReader()) {
>     return 0;
>   } else {
>     List readers = new ArrayList();
>     gatherSubReaders(readers);
>     Iterator iter = readers.iterator();
>     int maxDoc = 0;
>     while (iter.hasNext()) {
>       IndexReader r = (IndexReader);
>       if (r == reader) {
>         return maxDoc;
>       } 
>       maxDoc += r.maxDoc();
>     } 
>   }
>   return -1; // reader not in searcher
> }
> {code}
> Notes:
> * This workaround makes it so you cannot serialize your custom Weight implementation

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message