lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Segment Count of my Index is greater than the Configured MergeFactor
Date Thu, 19 Jun 2014 12:58:11 GMT
On 6/19/2014 2:51 AM, RadhaJayalakshmi wrote:
> I am using Solr 4.5.1. In that i have created an Index 114.8 MB. Also i have
> the following index  configuration
>            <indexConfig>
> 		<maxIndexingThreads>8</maxIndexingThreads>
> 		<ramBufferSizeMB>100</ramBufferSizeMB>
> 		<mergeFactor>10</mergeFactor>
> 	</indexConfig>
> 
> I have given the ramBufferSizeMB of 100 and mergefactor of 10. So this
> means, that after indexing is completed. i should see <=10 segments. Thats
> my assumption and even documentation says that.
> 
> But, after the indexing is completed, i went into Solr Dashboard, and
> selected the collection, for which indexing is completed. It is showing a
> Segment count of 13. 
> How is this possible? As i have given mergefactor 0f 10, at any point of
> time, there should not be more than 9 segments in the index.
> 
> I want to understand why 13 segments are created in my index??
> Could appreciate if i can get response ASAP

Imagine the following scenario.  You start from a clean index and do
enough indexing to create ten little segments.  At that point, Solr will
merge these segments into one large segment.  Let's say that now you do
enough indexing to create ten more segments.  It won't do the merge when
you reach nine little segments and one large segment ... it will do the
merge when you have ten little segments.  When the merge is done, you'll
be left with two large segments.  If you do enough indexing now to
create twenty new segments, then at the end you'll be left with four
large segments.  After this, if you index nine new segments, you've got
thirteen segments in your index and it won't do any more merging until
another segment is created.

Additional merge levels exist.  When you reach ten large segments, Solr
will merge those into one huge segment.  If indexing continues long
enough to create ten huge segments, they will be merged into one
enormous segment.  It would be possible to have a stable index with 9
segments at each of the levels that I have mentioned -- 36 segments.
The merge policy that Solr uses by default will continue creating
additional merge levels until the segments at the highest reach at least
five gigabytes in size -- nothing larger will be created unless you
optimize the index.

My effective merge factor is 35.  I have personally witnessed stable
indexes on my system with 80 or 90 segments.

Thanks,
Shawn


Mime
View raw message