lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: timing merges
Date Thu, 12 Jun 2014 14:51:06 GMT
1000 is way too high because it will mean your index has 1000s of
segments and when a merge does run it will take a very long time.
It's better to do smaller more frequent merges.  Try setting
segmentsPerTier to 5.

It's possible you are hitting too big a merge backlog, in which case
the default MergeScheduler (CMS) will pause incoming indexing threads,
letting merges catch up.  You could try increasing CMS.maxMergeCount
to allow a somewhat bigger backlog.  If your index is on an SSD you
should also increase CMS.maxThreadCount to 3 and maxMergeCount to
maybe 5 or 6.

Mike McCandless

http://blog.mikemccandless.com


On Thu, Jun 12, 2014 at 10:12 AM, Jamie <jamie@mailarchiva.com> wrote:
> Erick
>
> Well, I have users complaining about it. They say indexing stops for a long
> time.
>
> Currently, the following settings are applied.
>
> TieredMergePolicy logMergePolicy = new TieredMergePolicy();
> logMergePolicy.setSegmentsPerTier(1000);
>  conf.setMergePolicy(logMergePolicy);
>
> What's a good way to resolve this?
>
> Regards
>
> Jamie
>
>
> On 2014/06/12, 4:04 PM, Erick Erickson wrote:
>>
>> Michael is, of course, the Master of Merges...
>>
>> I have to ask, though, have you demonstrated to your satisfaction that
>> you're actually seeing a problem? And that fewer merges would actually
>> address that problem?
>>
>> 'cause this might be an "XY" problem....
>>
>> Best,
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message