Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of
 tamara.bobic@scai.fraunhofer.de designates 153.96.1.56 as permitted sender)
Date: Wed, 19 Oct 2011 13:18:34 +0200 (CEST)
From: Tamara Bobic <tamara.bobic@scai.fraunhofer.de>
To: java-user@lucene.apache.org, Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Subject: Re: OutOfMemoryError
Message-ID: <966afa13-62d7-4f9d-ac53-bb3f6eb6a021@zimbra.scai.fraunhofer.de>
In-Reply-To: <1318972452.51425.YahooMailNeo@web130120.mail.mud.yahoo.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Thank you all (Otis, Mead, Uwe) for your replies! 

It was very helpful and the problem turned out to be very trivial. I was ru=
nning 32-bit java instead of 64-bit and not enough memory could be reserved=
. 

Thanks once again, I finally managed to do the whole run successfully :)

All the best,
Tamara

----- Original Message -----
> From: "Otis Gospodnetic" <otis_gospodnetic@yahoo.com>
> To: java-user@lucene.apache.org
> Sent: Tuesday, October 18, 2011 11:14:12 PM
> Subject: Re: OutOfMemoryError
> 
> Bok Tamara,
> 
> You didn't say what -Xmx value you are using. =C2=A0Try a little higher
> value. =C2=A0Note that loading field values (and it looks like this one
> may be big because is compressed) from a lot of hits is not
> recommended.
> 
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
> 
> 
> >________________________________
> >From: Tamara Bobic <tamara.bobic@scai.fraunhofer.de>
> >To: java-user@lucene.apache.org
> >Cc: Roman Klinger <roman.klinger@scai.fraunhofer.de>
> >Sent: Tuesday, October 18, 2011 12:21 PM
> >Subject: OutOfMemoryError
> >
> >Hi all,
> >
> >I am using Lucene to query Medline abstracts and as a result I get
> >around 3 million hits. Each of the hits is processed and
> >information from a certain field is used.
> >
> >After certain number of hits, somewhere around 1 million (not always
> >the same number) I get OutOfMemory exception that looks like this:
> >
> >Exception in thread "main" java.lang.OutOfMemoryError
> >=C2=A0=C2=A0=C2=A0 at java.util.zip.Inflater.inflateBytes(Native Method)=

> >=C2=A0=C2=A0=C2=A0 at java.util.zip.Inflater.inflate(Inflater.java:221)
> >=C2=A0=C2=A0=C2=A0 at java.util.zip.Inflater.inflate(Inflater.java:238)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.document.CompressionTools.decompres=
s(CompressionTools.java:108)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.FieldsReader.uncompress(Field=
sReader.java:609)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.FieldsReader.addField(FieldsR=
eader.java:385)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.FieldsReader.doc(FieldsReader=
.java:231)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.SegmentReader.document(Segmen=
tReader.java:1013)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.DirectoryReader.document(Dire=
ctoryReader.java:520)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.FilterIndexReader.document(Fi=
lterIndexReader.java:149)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.index.IndexReader.document(IndexRea=
der.java:947)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.search.IndexSearcher.doc(IndexSearc=
her.java:152)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 org.apache.lucene.search.MultiSearcher.doc(MultiSearc=
her.java:156)
> >=C2=A0=C2=A0=C2=A0 at org.apache.lucene.search.Hits.doc(Hits.java:180)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 de.fhg.scai.bio.tamara.corpusBuilding.LuceneCmdLineIn=
terface.queryMedline(LuceneCmdLineInterface.java:178)
> >=C2=A0=C2=A0=C2=A0 at
> >=C2=A0=C2=A0=C2=A0 de.fhg.scai.bio.tamara.corpusBuilding.LuceneCmdLineIn=
terface.main(LuceneCmdLineInterface.java:152)
> >
> >
> >this line which causes problems is:
> >String docText =3D hits.doc(j).getField("DOCUMENT").stringValue() ;
> >
> >I am using java 1.6 and I tried solving this issue with different
> >garbage collectors (-XX:+UseParallelGC and -XX:+UseParallelOldGC)
> >but it didn't help.
> >
> >Does anyone have any idea how to solve this problem?
> >
> >There is also an official bug report:
> >http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D6293787
> >
> >Help is much appreciated. :)
> >
> >Best regards,
> >Tamara Bobic
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org