lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stanislav Jordanov <ste...@sirma.bg>
Subject Re: OutOfMemory when indexing
Date Tue, 14 Jun 2005 07:37:52 GMT
Thanks for the advice,
I think this may be a solution.
In case you've experimented with this setting, could you please tell me 
what are the side effects of limitting segment size?
This will probably cause searches to run slower?

Markus Wiederkehr wrote:

>I am not an expert, but maybe the occasionally high memory usage is
>because Lucene is merging multiple index segments together.
>
>Maybe it would help if you set maxMergeDocs to 10,000 or something. In
>your case that would mean that the minimum number of index segments
>would be 50.
>
>But again, this may be completely wrong...
>
>Markus
>
>On 6/13/05, Stanislav Jordanov <stenly@sirma.bg> wrote:
>  
>
>>High guys,
>>Building some huge index (about 500,000 docs totaling to 10megs of plain
>>text) we've run into the following problem:
>>Most of the time the IndexWriter process consumes a fairly small amount
>>of memory (about 32 megs).
>>However, as the index size grows, the memory usage sporadically bursts
>>to levels of (say) 1000 gigs and then falls back to its level.
>>The problem is that unless te process is started with some option like
>>-Xmx1000m this situation causes an OutOfMemoryException which terminates
>>the indexing process.
>>
>>My question is - is there a way to avoid it?
>>
>>Regards
>>Stanislav
>>    
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>  
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message