lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <>
Subject Re: Consolidate MP and LMP
Date Thu, 02 Dec 2010 09:43:11 GMT
Shai, I always have trouble to figure our what you are talking about
in the first place since you are never introducing the abbreviations
you are using :)

->> Consolidate MP and LMP  (WTF) :)

On Thu, Dec 2, 2010 at 10:33 AM, Michael McCandless
<> wrote:
> Hmm... but MegePolicy is our abstract base class, and LogMergePolicy
> adds alot of concrete stuff, ie choosing merges according to "level"
> so that you get an exponential staircase of segments in your index.
ah MergePolicy :)

> Conceptually it seems like they are separate?  Like if another merge
> policy came along that had the freedom to pick variable-mergeFactor
> segments to merge at once... shouldn't it subclass MP?  Or, say we fix
> IW to allow out-of-order merges, shouldn't that also subclass MP and
> not LMP?
I agree we should stick to that but requiring LMP in IW should be
fixed really. I actually see no reason why we should rely on
LogMergePolicy in IW and it should be easy to move the required
methods up  to the MergePolicy..
> I do agree IW requires LMP in some places, but isn't this limited to
> certain methods, eg set/getUseCompoundFile?  Maybe we can move just
> these methods up?

During the work on Column Stride Fields I was actually thinking that
Compound vs. Non-Compound should not be a global decision since we now
have codecs and each codec should use its own way of writing files.
Maybe it would make things way easier if we expose CFS to codecs and
let them decide what to do. I can imagine that I want to use CFS for
some of the codecs like Column Stride or fields that are not  used for
searches but keep individual files per codec. Just an idea....

> Mike
> On Thu, Dec 2, 2010 at 4:25 AM, Shai Erera <> wrote:
>> Hi
>> While IndexWriter declares it accepts a general MP, it will actually fail if
>> the given instance is not LogMP. So I wonder if we shouldn't consolidate
>> both of them into one, and pull up all of LMP features to MP. I think all of
>> LMP's features are useful for any kind of MP, and if someone wants to ignore
>> them he still can.
>> This is not the sort of change that fits well in trunk. IMO it can fit well
>> in 3x too since IW didn't accept anything that is not LMP. So even if it
>> will appear we're breaking back-compat, we actually won't. Which is another
>> reason, for me, why those two should be consolidated.
>> What do you think?
>> Shai
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message