lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-1483) Change IndexSearcher to use MultiSearcher semantics for sorted searches
Date Tue, 09 Dec 2008 19:37:44 GMT


Michael McCandless commented on LUCENE-1483:


OK I think I see what could explain this: insertion into the pqueue is
fairly costly.  So, because we now make 100 pqueues, each gathering
top N results, we are paying much more insertion cost overall than the
single queue that IndexSearcher(MultiReader) uses.

So.... how about still doing the searches per-sub-reader(searcher),
but, make a HitCollector that gathers the results into a single
pqueue, passing that HitCollector to each sub-searcher?

If that turns out OK, then I think it would make LUCENE-1471 moot
because we should similarly change MultiSearcher to use a single
shared pqueue.

Actually I think this approach should be a bit faster, because there
is some very small method call overhead to how MultiReader implements
TermDocs/Positions by "concatenating" its sub-readers.  So by pushing
Searcher down onto each SegmentReader we should gain a bit, but it
could very well be in the noise.  For this reason we may in fact want
to do this same thing for the "normal" (sort by relevance)

I wish I thought of this sooner.  Sorry for the runaround Mark!

> Change IndexSearcher to use MultiSearcher semantics for sorted searches
> -----------------------------------------------------------------------
>                 Key: LUCENE-1483
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>    Affects Versions: 2.9
>            Reporter: Mark Miller
>            Priority: Minor
>         Attachments: LUCENE-1483.patch
> Here is a quick test patch. FieldCache for sorting is done at the individual IndexReader
level and reloading the fieldcache on reopen can be much faster as only changed segments need
to be reloaded.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message