lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2694) MTQ rewrite + weight/scorer init should be single pass
Date Sun, 21 Nov 2010 12:04:14 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934260#action_12934260
] 

Michael McCandless commented on LUCENE-2694:
--------------------------------------------

BTW, one use case where this patch should show a sizable performance gain is a "primary key
lookup" against a multi-segment index.

So this'd be a TermQuery against eg an "id" field, where the app knows at most one doc contains
the requested value.

Today, we pay a high price for the 2nd pass, because we do not cache a miss against a segment.
 So on the first pass (computing IDF) we know which segment has a match and which segments
do not, but then on the 2nd pass we re-pay the lookup cost against all the misses (the single
segment w/ the hit will be cached).

So this ought to be a big win... especially once we combine this w/ the speedups from pulsing
codec (we still need to cutover to this as a default) then primary key lookups in a Lucene
index will be much faster...

> MTQ rewrite + weight/scorer init should be single pass
> ------------------------------------------------------
>
>                 Key: LUCENE-2694
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2694
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-2694.patch, LUCENE-2694.patch
>
>
> Spinoff of LUCENE-2690 (see the hacked patch on that issue)...
> Once we fix MTQ rewrite to be per-segment, we should take it further and make weight/scorer
init also run in the same single pass as rewrite.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message