Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: error (athena.apache.org: local policy)
MIME-Version: 1.0
Date: Fri, 9 Jan 2015 16:04:11 -0500
Message-ID: 
 <CAMySt+FjP2u8htbFRn3bDpKdc3GD2ZPBM5ZfmHeq0a=tZ8z5Qw@mail.gmail.com>
Subject: index writer closes due to OOM/heap space issue but no recovery after
 GC
From: Tom Burton-West <tburtonw@umich.edu>
To: java-user@lucene.apache.org
Content-Type: multipart/mixed; boundary=047d7b5d57369431a0050c3e7f94

--047d7b5d57369431a0050c3e7f94
Content-Type: multipart/alternative; boundary=047d7b5d573694319a050c3e7f92

--047d7b5d573694319a050c3e7f92
Content-Type: text/plain; charset=UTF-8

Hello,

I'm testing Solr 4.10.2 with 4GB allocated to the heap.  During the
indexing process I get an error message that says it is caused by an
"already closed indexwriter" due to an OOM. (See below).   After this
occurs it looks like the GC kicks in and there is plenty of heap space(see
attached)  but I continue getting this error.

Can someone please explain why after the GC frees memory, I continue to get
the error?

p.s. My documents average about 800KB and at completion each shard has over
3 billion unique terms.

Tom Burton-West


--------------------------------------------------
 org.apache.solr.common.SolrException;
org.apache.solr.common.SolrException: Exception writing document id
pst.000052087387 to the index; possible analysis error.        at
og.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:168)
caused by:
 org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
        at
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:698)

caused by
Caused by: java.lang.OutOfMemoryError: Java heap space
        at
org.apache.lucene.index.FreqProxTermsWriterPerField$FreqProxPostingsArray.<init>(FreqProxTermsWriterPerField.java:212)

--047d7b5d573694319a050c3e7f92
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello,<div><br></div><div>I&#39;m testing Solr 4.10.2 with=
 4GB allocated to the heap.=C2=A0 During the indexing process I get an erro=
r message that says it is caused by an &quot;already closed indexwriter&quo=
t; due to an OOM. (See below). =C2=A0 After this occurs it looks like the G=
C kicks in and there is plenty of heap space(see attached) =C2=A0but I cont=
inue getting this error.</div><div><br></div><div>Can someone please explai=
n why after the GC frees memory, I continue to get the error? =C2=A0=C2=A0<=
/div><div><br></div><div>p.s. My documents average about 800KB and at compl=
etion each shard has over 3 billion unique terms. =C2=A0</div><div><br></di=
v><div>Tom Burton-West</div><div><br></div><div><br></div><div><br></div><d=
iv>--------------------------------------------------</div><div><div>=C2=A0=
org.apache.solr.common.SolrException; org.apache.solr.common.SolrException:=
 Exception writing document id pst.000052087387 to the index; possible anal=
ysis error. =C2=A0 =C2=A0 =C2=A0 =C2=A0at og.apache.solr.update.DirectUpdat=
eHandler2.addDoc(DirectUpdateHandler2.java:168)</div></div><div>caused by:<=
br></div><div><div>=C2=A0org.apache.lucene.store.AlreadyClosedException: th=
is IndexWriter is closed</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apach=
e.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:698)</div></div><div=
><br></div><div>caused by</div><div><div>Caused by: java.lang.OutOfMemoryEr=
ror: Java heap space</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.lu=
cene.index.FreqProxTermsWriterPerField$FreqProxPostingsArray.&lt;init&gt;(F=
reqProxTermsWriterPerField.java:212)</div></div><div><br></div><div><br></d=
iv><div><br></div><div><br></div></div>

--047d7b5d573694319a050c3e7f92--

--047d7b5d57369431a0050c3e7f94
Content-Type: text/plain; charset=us-ascii


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
--047d7b5d57369431a0050c3e7f94--