lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: index writer closes due to OOM/heap space issue but no recovery after GC
Date Sat, 10 Jan 2015 10:01:26 GMT
IW closes itself on "tragic" events like OOME to guard against index
corruption....

Mike McCandless

http://blog.mikemccandless.com


On Fri, Jan 9, 2015 at 4:04 PM, Tom Burton-West <tburtonw@umich.edu> wrote:
> Hello,
>
> I'm testing Solr 4.10.2 with 4GB allocated to the heap.  During the indexing
> process I get an error message that says it is caused by an "already closed
> indexwriter" due to an OOM. (See below).   After this occurs it looks like
> the GC kicks in and there is plenty of heap space(see attached)  but I
> continue getting this error.
>
> Can someone please explain why after the GC frees memory, I continue to get
> the error?
>
> p.s. My documents average about 800KB and at completion each shard has over
> 3 billion unique terms.
>
> Tom Burton-West
>
>
>
> --------------------------------------------------
>  org.apache.solr.common.SolrException; org.apache.solr.common.SolrException:
> Exception writing document id pst.000052087387 to the index; possible
> analysis error.        at
> og.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:168)
> caused by:
>  org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
>         at
> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:698)
>
> caused by
> Caused by: java.lang.OutOfMemoryError: Java heap space
>         at
> org.apache.lucene.index.FreqProxTermsWriterPerField$FreqProxPostingsArray.<init>(FreqProxTermsWriterPerField.java:212)
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message