lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Consolidate MP and LMP
Date Thu, 02 Dec 2010 11:17:43 GMT
On Thu, Dec 2, 2010 at 6:10 AM, Earwin Burrfoot <earwin@gmail.com> wrote:

> What the hell is mergeFactor?! Right now this LMP setting serves two
> completely separate roles at once.
> 1. it governs a maximum number of segments on a "level";
> 2. it also limits a maximum number of segments that can participate in
> a single merge (thus throttling IO randomness).

I agree these are two concepts.

We could make a MP that lets you separately control max number of
segments on a level vs how many segments are merged at once.

So eg if I allow 20 segments per level but merge 10 at once, the first
merge would happen after 20 flushed segments, but would then leave 11
segments in the index.

One nice property of such a MP is it would never "inadvertently
optimize" your index, which is rather wasteful pay-it-forward approach
that our LMP does today that causes your index to sometimes be in an
"over merged" state.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message