lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: MergePolicy Thresholds
Date Tue, 03 May 2011 09:29:49 GMT
Hi

I looked into porting it to 3x, and prepared the attached patch. It only
contains the new TieredMP and Test, as well as the necessary changes to
LuceneTestCase and IndexWriter. I guess you can start with it (even just the
MP and IW changes) to test it on your indexes.

Mike, I saw that there were many more changes, as part of LUCENE-1076, done
to the code. In particular, this MP is now the default (on trunk), so I
guess many changes (to tests) were needed because of that. Do you remember,
if apart from the changes I've included in the patch, other important
changes w.r.t. this code?

As we won't change the default MP on 3x, I'm guessing I don't need to port
all the changes to 3x.

Shai

On Mon, May 2, 2011 at 9:41 PM, Burton-West, Tom <tburtonw@umich.edu> wrote:

> Hi Shai and Mike,
>
> Testing the TieredMP on our large indexes has been on my todo list since I
> read Mikes blog post
>
> http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
> .
>
> If you port it to the 3.x branch Shai, I'll be more than happy to test it
> with our very large (300GB+) indexes.  Besides being able to set the max
> merged segment size, I'm especially interested in using the
>  maxSegmentsPerTier parameter.
>
> From Mike's blog post:
> " ...maxSegmentsPerTier that lets you set the allowed width (number of
> segments) of each stair in the staircase. This is nice because it decouples
> how many segments to merge at a time from how wide the staircase can be."
>
> Tom Burton-West
> http://www.hathitrust.org/blogs/large-scale-search
>
> -----Original Message-----
> From: Michael McCandless [mailto:lucene@mikemccandless.com]
> Sent: Monday, May 02, 2011 2:19 PM
> To: dev@lucene.apache.org
> Subject: Re: MergePolicy Thresholds
>
> I think it should be an easy port...
>
> Mike
>
> http://blog.mikemccandless.com
>
> On Mon, May 2, 2011 at 2:16 PM, Shai Erera <serera@gmail.com> wrote:
> > Thanks Mike. I'll take a look at TieredMP. Does it depend on trunk in any
> > way, or do you think it can easily be ported to 3x?
> > Shai
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message