lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: Consolidate MP and LMP
Date Thu, 02 Dec 2010 11:28:30 GMT
On Thu, Dec 2, 2010 at 12:14 PM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> On Thu, Dec 2, 2010 at 4:43 AM, Simon Willnauer
> <simon.willnauer@googlemail.com> wrote:
>
>> During the work on Column Stride Fields I was actually thinking that
>> Compound vs. Non-Compound should not be a global decision since we now
>> have codecs and each codec should use its own way of writing files.
>> Maybe it would make things way easier if we expose CFS to codecs and
>> let them decide what to do. I can imagine that I want to use CFS for
>> some of the codecs like Column Stride or fields that are not  used for
>> searches but keep individual files per codec. Just an idea....
>
> +1!
>
> This would be a nice simplification.

I created an issue for this: LUCENE-2789


simon
>
> EG, it's bizarre today that on flushing a new segment, which has
> nothing to do with merging, we consult the MP to decide if we need CFS
> or not.
>
> Also, it's awkward we have getCF and also getCFDocStore.  In the
> future (docvalues) we may also want to separately build CFS for those
> files, or not.
>
> Making all these decisions private to the codec makes great sense.
> It's then free to CFS however it wants to.  But, the codec would need
> wider context, I think the full SegmentInfos, to base its decision on.
>  EG, LMP now conditionally builds CFS only if the segment is
> "smallish" relative to total index size.
>
> Mike
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message