lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-2694) MTQ rewrite + weight/scorer init should be single pass
Date Sun, 21 Nov 2010 12:04:14 GMT


Michael McCandless commented on LUCENE-2694:

BTW, one use case where this patch should show a sizable performance gain is a "primary key
lookup" against a multi-segment index.

So this'd be a TermQuery against eg an "id" field, where the app knows at most one doc contains
the requested value.

Today, we pay a high price for the 2nd pass, because we do not cache a miss against a segment.
 So on the first pass (computing IDF) we know which segment has a match and which segments
do not, but then on the 2nd pass we re-pay the lookup cost against all the misses (the single
segment w/ the hit will be cached).

So this ought to be a big win... especially once we combine this w/ the speedups from pulsing
codec (we still need to cutover to this as a default) then primary key lookups in a Lucene
index will be much faster...

> MTQ rewrite + weight/scorer init should be single pass
> ------------------------------------------------------
>                 Key: LUCENE-2694
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>         Attachments: LUCENE-2694.patch, LUCENE-2694.patch
> Spinoff of LUCENE-2690 (see the hacked patch on that issue)...
> Once we fix MTQ rewrite to be per-segment, we should take it further and make weight/scorer
init also run in the same single pass as rewrite.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message