lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <>
Subject Re: Consolidate MP and LMP
Date Thu, 02 Dec 2010 11:28:30 GMT
On Thu, Dec 2, 2010 at 12:14 PM, Michael McCandless
<> wrote:
> On Thu, Dec 2, 2010 at 4:43 AM, Simon Willnauer
> <> wrote:
>> During the work on Column Stride Fields I was actually thinking that
>> Compound vs. Non-Compound should not be a global decision since we now
>> have codecs and each codec should use its own way of writing files.
>> Maybe it would make things way easier if we expose CFS to codecs and
>> let them decide what to do. I can imagine that I want to use CFS for
>> some of the codecs like Column Stride or fields that are not  used for
>> searches but keep individual files per codec. Just an idea....
> +1!
> This would be a nice simplification.

I created an issue for this: LUCENE-2789

> EG, it's bizarre today that on flushing a new segment, which has
> nothing to do with merging, we consult the MP to decide if we need CFS
> or not.
> Also, it's awkward we have getCF and also getCFDocStore.  In the
> future (docvalues) we may also want to separately build CFS for those
> files, or not.
> Making all these decisions private to the codec makes great sense.
> It's then free to CFS however it wants to.  But, the codec would need
> wider context, I think the full SegmentInfos, to base its decision on.
>  EG, LMP now conditionally builds CFS only if the segment is
> "smallish" relative to total index size.
> Mike

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message