lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: FieldSelector
Date Wed, 05 Dec 2007 11:20:51 GMT

On Dec 5, 2007, at 2:40 AM, Timo Nentwig wrote:

> On Friday 30 November 2007 19:28:12 Grant Ingersoll wrote:
>> I guess the question becomes what is the nature of your fields?  Do
>> you have some really large fields that you want to avoid loading b/c
>> they are not shown initially?  That is the main use case, I guess.
>
> I wonder why there's not LazyFieldSelector that loads all fields  
> lazyly in
> general. Is this for some reason slow or why is this obviously  
> reasonable
> class missing?

Probably because it wasn't obvious to me :-)  That is, my use case was  
almost always to load small fields right away and large fields  
lazily.  I haven't tested the cost of this, but you would essentially  
be doing all the work and then some to load them all lazily, so I am  
not sure I see the point.  For example, if you have 4 fields, 3 small  
and 1 large, the FieldsReader more or less does:

1. move the pointer to the start of the doc
2. Check the field selector
3. if Lazy, store a pointer, else load the field or skip
4. Skip ahead to the next field, repeat 2-4 until all fields in the  
doc have been addressed

Then, when you go to access those 4 fields, which you most certainly  
will at some point soon, otherwise why did you get the document to  
begin with, it then has to go reset the pointer to that position and  
then load the info.  Chances are, b/c of the way OSes handle disk,  
etc. you've already loaded all the small info (and probably in some  
cases the big data too, depending on size) so why not just return it  
the first time?  In cases of really small fields (i.e. integers, etc.)  
the cost of storing the pointer is probably really close if not the  
same as storing the field, if not more b/c now you have to store the  
pointer plus do a seek.

Cheers,
Grant



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message