lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sat, 06 Dec 2008 20:33:11 GMT

Mark Miller wrote:

> Michael McCandless (JIRA) wrote:
>>  However, stepping back, this is poor approach.  We should instead be
>> doing what MultiSearcher does, which is gather top results
>> per-sub-reader, and then merge-sort the results.  At that point, to  
>> do
>> the merge, we only need actual field values for those docs in the top
>> N.
> What if one segment has the 20 best docs and a second segment has  
> only pretty poor docs - and then various mixes in between. This is  
> how it works with MultiSearcher right?

I was thinking only about sorting by field(s), which I think is  
identical through MultiSearcher vs IndexSearcher(MultiSegmentReader).

Today, with IndexSearcher(MultiReader), the FieldSortedHitQueue asks
FieldCache to materialize the full array for each field.  Whereas
MultiSearcher only asks each child reader to materialize its array for
the field, which is better because on reopen we only need to init the
array for the new segments.

If somehow we could make IndexSearcher(MultiReader) implement it's
field sorting searchers the same way MultiSearcher does, I think
that'd be a big win.

> MultiSearcher has a few aspects I don't like.

Do you mean the score differences vs IndexSearcher(MultiReader), or is  
there something else?

> I may not be fully understanding yet though. I have a slow  
> understanding time and a fast response time. Brilliant  
> combination :) Its like a fish with feet.

I too don't fully understand it!  Trying to think it through still...

> I hope I'll be playing around more with some of these ideas though  
> (free time is not as plentiful as I would like at the moment). I  
> should have some better comments to make about it then.

It'd be great to get closure on LUCENE-831.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message