lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stefan" <>
Subject AW: OutOfMemoryError using IndexWriter
Date Wed, 24 Jun 2009 14:06:27 GMT


there seems to be a little misunderstanding. The index will only be optimized if the IndexWriter
is to be closed and then only with a probability of 2% (meaning occasionaly).
In other words, I only close the IndexWriter (and thus optimize) to avoid the OOMError. 
When I keep the same IndexWriter for the complete indexing operation, I do not call optimize
but get an OOMError.


-----Urspr√ľngliche Nachricht-----
Von: Otis Gospodnetic []
Gesendet: Mi 24.06.2009 14:22
Betreff: Re: OutOfMemoryError using IndexWriter

Hi Stefan,

While not directly th source of your problem, I have a feeling you are optimizing too frequently
(and wasting time/CPU by doing so).  Is there a reason you optimize so often?  Try optimizing
only at the end, when you know you won't be adding any more documents to the index for a while.

I'm also wondering why you close/open the IndeWriter, as it looks like you are doing batch

Sematext -- -- Lucene - Solr - Nutch

----- Original Message ----
> From: stefan <>
> To:
> Sent: Wednesday, June 24, 2009 4:08:43 AM
> Subject: OutOfMemoryError using IndexWriter
> Hi,
> I am using Lucene 2.4.1 to index a database with less than a million records. 
> The resulting index is about 50MB in size.
> I keep getting an OutOfMemory Error if I re-use the same IndexWriter to index 
> the complete database. This is though 
> recommended in the performance hints.
> What I now do is, every 10000 Objects I close the index (and every 50 close 
> actions optimize it) and create a new
> IndexWriter to continue. This process works fine, but to me seems hardly the 
> recommended way to go.
> I've been using jhat/jmap as well as Netbeans profiler and am fairly sure that 
> this is a problem related to Lucene.
> Any Ideas - or post this to Jira ? Jira has quite a few OutOfMemory postings but 
> they all seem closed in Version 2.4.1.
> Thanks,
> Stefan
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message