lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: ConcurrentMergeScheduler and MergePolicy question
Date Thu, 30 Jul 2009 14:48:40 GMT
Note also response from Mike that talks a little bit about something  
along these lines: http://www.lucidimagination.com/search/document/fa990adba4d2572b/is_there_a_way_to_control_when_merges_happen#f6f0bfeef4bf9a39

-Grant

On Jul 30, 2009, at 10:35 AM, Grant Ingersoll wrote:

> Given a large segment and a bunch of small segments, how does the  
> ConcurrentMergeScheduler (CMS) work?  Does it always merge the  
> smaller segments into the bigger one, or does it merge the smaller  
> segments together?
>
> Something I've been thinking about:  Given a high update environment  
> (and near real time, less than 1 minute, search constraints) and/or  
> a very bursty environment, we've always said to keep the merge  
> factor small for search reasons, at least in the high-update case.   
> However, I've seen a couple of times where this causes problems  
> because merges can take over and cause pauses, even with CMS, so I  
> am wonder if it makes sense to have a larger merge factor (>10),  
> knowing that I may have a few large segments and then a bunch of  
> small ones and that the CMS will, in the background, be able to keep  
> merging the smaller segments together and in most cases avoid ever  
> having to merge into the large segments (b/c maybe I can just  
> optimize down at slower times or even merge larger segments  
> later. )   Seems like this would allow one to make sure larger  
> merges need not take place, or at least reduce the chances of that  
> happening.
>
> Not sure if I worded that correctly.
>
> Thanks,
> Grant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message