lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: OutOfMemoryError using IndexWriter
Date Thu, 25 Jun 2009 10:01:44 GMT
On Thu, Jun 25, 2009 at 3:02 AM, stefan<> wrote:

>>But a "leak" would keep leaking over time, right?  Ie even a 1 GB heap
>>on your test db should eventually throw OOME if there's really a leak.
> No not necessarily, since I stop indexing ones everything is indexed - I shall try repeated
runs with 120MB.

It indeed looks like IndexWriter doens't account for RAM used by
pending deletes.  I've opened for this.  Though
I'd normally expect the amount of extra RAM used to be smallish...

Do you have a high merge factor?  Can you run CheckIndex on your index
(java org.apache.lucene.index.CheckIndex /path/to/index) and post the
output back?

Currently IndexWriter will flush these deletes on kicking off a merge,
or if commit() is called, so one workaround you could try is to call
commit() every so often and see if that improves the RAM usage?

>>Are you calling updateDocument (which deletes then adds)?
> Yes I do, I do not know in my code whether the document is already indexed or not. In
my test case I do delete the
> complete index before the run, so all documents should be new to the index. I still use
update though, since I
> this piece of code is generic.

OK that's a good reason to use updateDocument.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message