lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kireet Reddy <kir...@feedly.com>
Subject TieredMergePolicy
Date Sun, 06 Jul 2014 01:47:28 GMT
I have a question about the maxMergeAtOnce parameter. We are using elasticsearch and one of
our nodes seems to have very high merge activity, However it seems to be high CPU activity
and not I/O constrainted. I have enabled the IndexWriter info stream logs, and often times
it seems to do merges of quite small segments (100KB) that are much below the floor size (2MB).
I suspect this is due to frequent refreshes and/or using lots of threads concurrently to do
indexing. 

My supposition is that this is leading to the merge policy doing lots of merges of very small
segments into another small segment which will again require a merge to even reach the floor
size. My index has 64 segments and 25 are below the floor size. I am wondering if there should
be an exception for the maxMergesAtOnce parameter for the first level so that many small segments
could be merged at once in this case?

I am considering changing the other parameters (wider tiers, lower floor size, more concurrent
merges allowed) but these all seem to have side effects I may not necessarily want. Is there
a good solution here?
Mime
View raw message