lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sun, 07 Dec 2008 09:38:02 GMT

Mark Miller wrote:

> Michael McCandless wrote:
>> Today, with IndexSearcher(MultiReader), the FieldSortedHitQueue asks
>> FieldCache to materialize the full array for each field.  Whereas
>> MultiSearcher only asks each child reader to materialize its array  
>> for
>> the field, which is better because on reopen we only need to init the
>> array for the new segments.
> Thats what the ObjectArray does.


> There are currently two approaches in the patch.
> The primitive array approach is the one that uses the merge to  
> create a larger array. The win here is only on loading from disk  
> with reopen.
> The other approach is the ObjectArray. Its win is no merge and each  
> sub reader keeps its own array with no super array being created.  
> The loses right now are the binarysearch of subreaders, the method  
> call, and the cast. Seems to be about a 5-10% slowdown (if I  
> remember right).

I'd like to decouple "upgraded to Object" vs "materialize full array",  
ie, so we can access native values w/o materializing the full array.   
I also think "upgrade to Object" is dangerous to even offer since it's  
so costly.

> You have to choose one of the two 'mode' of operation with the  
> system property. This is not the type of thing that I want to end up  
> in the patch. However, it was even uglier a few iterations ago, so  
> eventually...I just keep pushing a little every few months when it  
> catches my eye. There is too much involved here for me to handle  
> myself (in terms of decisions, design), and prior to now, the main  
> interest has been by watchers.



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message