Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Content-class: urn:content-classes:message
Subject: AW: OutOfMemoryError using IndexWriter
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----_=_NextPart_001_01C9F4D6.AA1E3119"
Date: Wed, 24 Jun 2009 16:18:37 +0200
Message-ID: <E28B87DDBA121F45B476C053B2171F0E596BDB@mail.intermediate.de>
Thread-Topic: OutOfMemoryError using IndexWriter
Thread-Index: Acn0y27fb8h8WF+aS3e6lzUBHBrzegACgxb3
References: 
 <E28B87DDBA121F45B476C053B2171F0E596BD6@mail.intermediate.de><9ac0c6aa0906240148ybcd0694ua3b9f7b7788a1c4b@mail.gmail.com><E28B87DDBA121F45B476C053B2171F0E596BD7@mail.intermediate.de><9ac0c6aa0906240252v36bb1301u6abdd7e0d5ffbd56@mail.gmail.com><E28B87DDBA121F45B476C053B2171F0E596BD8@mail.intermediate.de>
 <9ac0c6aa0906240557l616e9f7ch2b1067398d2f1082@mail.gmail.com>
From: "stefan" <stefan@intermediate.de>
To: <java-user@lucene.apache.org>

------_=_NextPart_001_01C9F4D6.AA1E3119
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable


Hi,


>OK so this means it's not a leak, and instead it's just that stuff is
>consuming more RAM than expected.
Or that my test db is smaller than the production db which is indeed the =
case.

>Hmm -- there are quite a few buffered deletes pending.  It could be we
>are under-accounting for RAM used by buffered deletes.  I'll dig on
>that.
That sounds promising to me, how does a delete happen, we are talking of =
complete new re-indexing, no deletes at all should be happening. Are you =
saying that I remove docs from the index ?


>Also, your char[]'s are taking ~30 MB, byte[] ~26MB, which is odd if
>your RAM buffer is 16MB.  Does your app create these?
A fair amount is created by my app - I histogram without indexing shows =
about 10MB chars created by my App:
class [C 	132399 	9457148,
though a few more could be created during indexing.

> Why is it, that creating a new Index Writer will let the indexing run =
fine with 80MB, but keeping it will create an
> OutOfMemoryException running with 100MB heap size ?

Please explain those buffered deletes in a few more details.

Thanks,

Stefan


------_=_NextPart_001_01C9F4D6.AA1E3119
Content-Type: text/plain; charset=us-ascii


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
------_=_NextPart_001_01C9F4D6.AA1E3119--