lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Mon, 08 Dec 2008 11:05:31 GMT

On thinking more about this... I think with a few small changes we
could achieve Sort by field without materializing a full array.  We
can decouple this change from LUCENE-831.

I think all that's needed is:

   * Expose sub-readers (LUCENE-1475) by adding IndexReader[]
     IndexReader.getSubReaders.  Default impl could just return
     length-1 array of itself.

   * Change IndexSearcher.sort that takes a Sort, to first call
     IndexReader.getSubReaders, and then do the same logic that
     MultiSearcher does, with improvements from LUCENE-1471 (run
     separate search per-reader, then merge-sort the top hits from

The results should be functionally identical to what we have today,
but, searching after doing a reopen() should be much faster since we'd
no longer re-build the global FieldCache array.

Does this make sense?  It's a small change for a big win, I think.
Does anyone want to take a crack at this patch?


Mark Miller wrote:

> Michael McCandless wrote:
>> I'd like to decouple "upgraded to Object" vs "materialize full  
>> array", ie, so we can access native values w/o materializing the  
>> full array.  I also think "upgrade to Object" is dangerous to even  
>> offer since it's so costly.
> I'm right with you. I didn't think the Object approach was really an  
> upgrade (beyond losing the merge, which is especially important for  
> StringIndex - it has no merge option at the moment) which is why I  
> left both options for now. So I def agree we need to move to  
> iterator, drop object, etc.
> Its the doin' that aint so easy. The iterator approach seems  
> somewhat straightforward (though its complicated by needing to  
> provide a random access object as well), but I'm still working  
> through how we control so many iterator types (I dont see how you  
> can use polymorphism yet ).
> - Mark
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message