lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Separate issue for appendable field caches / doc values
Date Tue, 23 Aug 2011 16:30:45 GMT
On Tue, Aug 23, 2011 at 12:09 AM, Jason Rutherglen
<jason.rutherglen@gmail.com> wrote:
>> But, we are trying to move FC under IR control (Martijn has a patch),
>> at which point an RT IR could have its own appending impl...?
>
> LUCENE-3360?  It's placing the field cache into IR / SR.

Yes!

> The RAM
> reader could return it's own impl where the underlying array can be
> atomically replaced (when a larger sized array is needed).  I agree
> that is logical.

Good.

>> But then... FC still returns fixed arrays so you can't append until we fix that?
>
> I don't think anything should depend on the size of the field cache
> array.  If it does, it seems odd.  Are you preferring moving field
> cache access to method calls?  What is the difference between that and
> primitive array access?

Oh duh right -- you should be able to alloc too-large arrays and only
realloc once you run out.  So amortized cost is low but some reopens
will take a hit to grow....

> For now I will create an independent field cache implementation that
> is appendable.  It will only be associate-able with the DWPT / RAM
> reader.  Maybe somehow it can work with LUCENE-3360 and / or the
> existing static FC access system.

Sounds good.

> Still not sure how doc values fits in.

DV is also provided by the IR (perDocValues method) so the RT reader
should be able to impl its own.  Each lookup is a method call so it
should be easier to back that w/ a more RT friendly data structure...

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message