lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: Consolidate MP and LMP
Date Thu, 02 Dec 2010 11:14:42 GMT
On Thu, Dec 2, 2010 at 4:43 AM, Simon Willnauer
<> wrote:

> During the work on Column Stride Fields I was actually thinking that
> Compound vs. Non-Compound should not be a global decision since we now
> have codecs and each codec should use its own way of writing files.
> Maybe it would make things way easier if we expose CFS to codecs and
> let them decide what to do. I can imagine that I want to use CFS for
> some of the codecs like Column Stride or fields that are not  used for
> searches but keep individual files per codec. Just an idea....


This would be a nice simplification.

EG, it's bizarre today that on flushing a new segment, which has
nothing to do with merging, we consult the MP to decide if we need CFS
or not.

Also, it's awkward we have getCF and also getCFDocStore.  In the
future (docvalues) we may also want to separately build CFS for those
files, or not.

Making all these decisions private to the codec makes great sense.
It's then free to CFS however it wants to.  But, the codec would need
wider context, I think the full SegmentInfos, to base its decision on.
 EG, LMP now conditionally builds CFS only if the segment is
"smallish" relative to total index size.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message