lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Wiederkehr <markus.wiederk...@gmail.com>
Subject Re: OutOfMemory when indexing
Date Mon, 13 Jun 2005 13:30:50 GMT
I am not an expert, but maybe the occasionally high memory usage is
because Lucene is merging multiple index segments together.

Maybe it would help if you set maxMergeDocs to 10,000 or something. In
your case that would mean that the minimum number of index segments
would be 50.

But again, this may be completely wrong...

Markus

On 6/13/05, Stanislav Jordanov <stenly@sirma.bg> wrote:
> High guys,
> Building some huge index (about 500,000 docs totaling to 10megs of plain
> text) we've run into the following problem:
> Most of the time the IndexWriter process consumes a fairly small amount
> of memory (about 32 megs).
> However, as the index size grows, the memory usage sporadically bursts
> to levels of (say) 1000 gigs and then falls back to its level.
> The problem is that unless te process is started with some option like
> -Xmx1000m this situation causes an OutOfMemoryException which terminates
> the indexing process.
> 
> My question is - is there a way to avoid it?
> 
> Regards
> Stanislav

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message