lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation
Date Sun, 07 Dec 2008 09:38:02 GMT

Mark Miller wrote:

> Michael McCandless wrote:
>>
>> Today, with IndexSearcher(MultiReader), the FieldSortedHitQueue asks
>> FieldCache to materialize the full array for each field.  Whereas
>> MultiSearcher only asks each child reader to materialize its array  
>> for
>> the field, which is better because on reopen we only need to init the
>> array for the new segments.
> Thats what the ObjectArray does.

OK.

> There are currently two approaches in the patch.
>
> The primitive array approach is the one that uses the merge to  
> create a larger array. The win here is only on loading from disk  
> with reopen.
>
> The other approach is the ObjectArray. Its win is no merge and each  
> sub reader keeps its own array with no super array being created.  
> The loses right now are the binarysearch of subreaders, the method  
> call, and the cast. Seems to be about a 5-10% slowdown (if I  
> remember right).

I'd like to decouple "upgraded to Object" vs "materialize full array",  
ie, so we can access native values w/o materializing the full array.   
I also think "upgrade to Object" is dangerous to even offer since it's  
so costly.

> You have to choose one of the two 'mode' of operation with the  
> system property. This is not the type of thing that I want to end up  
> in the patch. However, it was even uglier a few iterations ago, so  
> eventually...I just keep pushing a little every few months when it  
> catches my eye. There is too much involved here for me to handle  
> myself (in terms of decisions, design), and prior to now, the main  
> interest has been by watchers.

OK.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message