lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stefan" <ste...@intermediate.de>
Subject AW: OutOfMemoryError using IndexWriter
Date Wed, 24 Jun 2009 14:06:27 GMT

Hi,

there seems to be a little misunderstanding. The index will only be optimized if the IndexWriter
is to be closed and then only with a probability of 2% (meaning occasionaly).
In other words, I only close the IndexWriter (and thus optimize) to avoid the OOMError. 
When I keep the same IndexWriter for the complete indexing operation, I do not call optimize
but get an OOMError.

Stefan



-----Urspr√ľngliche Nachricht-----
Von: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Gesendet: Mi 24.06.2009 14:22
An: java-user@lucene.apache.org
Betreff: Re: OutOfMemoryError using IndexWriter
 

Hi Stefan,

While not directly th source of your problem, I have a feeling you are optimizing too frequently
(and wasting time/CPU by doing so).  Is there a reason you optimize so often?  Try optimizing
only at the end, when you know you won't be adding any more documents to the index for a while.

I'm also wondering why you close/open the IndeWriter, as it looks like you are doing batch
indexing.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: stefan <stefan@intermediate.de>
> To: java-user@lucene.apache.org
> Sent: Wednesday, June 24, 2009 4:08:43 AM
> Subject: OutOfMemoryError using IndexWriter
> 
> Hi,
> 
> I am using Lucene 2.4.1 to index a database with less than a million records. 
> The resulting index is about 50MB in size.
> I keep getting an OutOfMemory Error if I re-use the same IndexWriter to index 
> the complete database. This is though 
> recommended in the performance hints.
> What I now do is, every 10000 Objects I close the index (and every 50 close 
> actions optimize it) and create a new
> IndexWriter to continue. This process works fine, but to me seems hardly the 
> recommended way to go.
> I've been using jhat/jmap as well as Netbeans profiler and am fairly sure that 
> this is a problem related to Lucene.
> 
> Any Ideas - or post this to Jira ? Jira has quite a few OutOfMemory postings but 
> they all seem closed in Version 2.4.1.
> 
> Thanks,
> 
> Stefan
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org




Mime
View raw message