lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: OutOfMemoryError using IndexWriter
Date Thu, 25 Jun 2009 13:57:02 GMT
Interesting that excessive deletes buffering is not your problem...

Even if you can't post the resulting test case, if you can simplify it
& run locally, to rule out anything outside Lucene that's allocating
the byte/char/byte[] arrays, that can help to isolate.

Also, profilers can trace where allocations happened, eg YourKit.

Mike

On Thu, Jun 25, 2009 at 9:08 AM, stefan<stefan@intermediate.de> wrote:
> Hi,
>
> I'm afraid my test setup and code this is far too big.
> What I use lucene for is fairly simple. I have a database with about 150 tables, I iterate
all tables and create for each row a String representation similar to a toString method containing
all database data. This string is then fed together with the primary key to lucene. Full-text
search of my db is then possible. Each document in Lucene represents a row in the database.
>
> I tried calling setMaxBufferedDeleteTerms  with 100MB heap size to no avail, but calling
commit every 10000 documents does help. I assume a commit is similar to creating a new IndexWriter.
>
> HTH,
>
> Stefan
>
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Michael McCandless [mailto:lucene@mikemccandless.com]
> Gesendet: Do 25.06.2009 13:13
> An: java-user@lucene.apache.org
> Betreff: Re: OutOfMemoryError using IndexWriter
>
> Can you post your test code?  If you can make it a standalone test,
> then I can repro and dig down faster.
>
> Can you try calling IndexWriter.setMaxBufferedDeleteTerms (eg, 1000)
> and see if that prevents the OOM?
>
> Mike
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message