lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: java.lang.IllegalArgumentException: Segment is too large
Date Mon, 31 Mar 2008 09:19:21 GMT

This happens when addIndexesNoOptimize finds a segment that's larger  
than maxMergeDocs in the index(es) it's given.

If you leave maxMergeDocs at Integer.MAX_VALUE it will fix that.

Though really it's being a little too pedantic because that setting  
(maxMergeDocs) sets the maximum size of a segment that's may be  
selected for merging.  So it's easily possible to have segments in  
your index that are larger than this (eg by merging segments that  
were all just a bit smaller than the limit).

I think we should remove those checks and allow addIndexesNoOptimize  
to import and index even if it has segments over this limit.  I'll  
open an issue.

Mike

vivek sar wrote:
> Hi,
>
>  I'm using 2.3.0 Lucene build and have following merge parameters,
>
> mergeFactor = 100
> maxMergeDocs = 99999
> maxBufferedDocs = 10000
> maxRAMBufferSizeMB = 200
>
>
> After running with this setting for a month without problem  all the
> sudden I'm getting following exception,
>
> java.lang.IllegalArgumentException: Segment is too large (100001 docs
> vs max docs 99999)
>         at org.apache.lucene.index.LogMergePolicy.findMerges 
> (LogMergePolicy.java:274)
>         at org.apache.lucene.index.IndexWriter.updatePendingMerges 
> (IndexWriter.java:1824)
>         at org.apache.lucene.index.IndexWriter.maybeMerge 
> (IndexWriter.java:1799)
>         at org.apache.lucene.index.IndexWriter.maybeMerge 
> (IndexWriter.java:1795)
>         at org.apache.lucene.index.IndexWriter.maybeMerge 
> (IndexWriter.java:1791)
>         at org.apache.lucene.index.IndexWriter.addIndexesNoOptimize 
> (IndexWriter.java:2231)
>
>
> I thought Lucene should control the number of documents in a segment
> during merge. Is there a different configuration I need to set? Should
> I leave the maxMergeDocs to its default value, Integer#MAX_VALUE?
>
> Thanks,
> -vivek
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message