lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <jason.rutherg...@gmail.com>
Subject Re: Is there a way to limit the size of an index?
Date Thu, 07 Jan 2010 22:21:40 GMT
The naming is unclear, when I looked at this I had to thumb
through the code a fair bit before discerning if it was the
input segments or the output segment of a merge (it's the
former). Though I find the current functionality somewhat odd
because it will inherently exceed the given size with a merge,
which is uncontrollable, making the exact setting somewhat
arbitrary... What was the original use case?

On Thu, Jan 7, 2010 at 11:23 AM, Otis Gospodnetic
<otis_gospodnetic@yahoo.com> wrote:
>> Merge factor controls how many segments are merged at once.  The default is 10.
>>
>> The maxMergeMB setting sets the max size for a given segment to be
>> included in a merge.
>
> I wonder if renaming that to maxSegSizeMergeMB would make it more obvious what this does?
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
>
>> Roughly, the upper bound on merged segments is the sum of their sizes.
>>
>> So the rough upper bound on any segment's size is mergeFactor * maxMergeMB.
>>
>> Mike
>>
>> On Thu, Jan 7, 2010 at 11:04 AM, Dvora wrote:
>> >
>> > Can you explain how the combination of merge factor and max merge size
>> > control the size of files?
>> >
>> > For example, if one would like to limit the files size to 3,4 or 7MB - how
>> > these parameters values can be predicted?
>> >
>> >
>> >
>> > Michael McCandless-2 wrote:
>> >>
>> >>
>> >> This tells the IndexWriter NOT to merge any segment that's over 1.0 MB
>> >> in size.  With a default merge factor of 10, this should "generally"
>> >> mean you don't get a segment over 10MB, though it may not be a hard
>> >> guarantee (you can lower the 1.0 if you still see a segment over 10
>> >> MB).
>> >>
>> >>
>> >
>> > --
>> > View this message in context:
>> http://old.nabble.com/Is-there-a-way-to-limit-the-size-of-an-index--tp27056573p27062291.html
>> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message