lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: OutOfMemoryError
Date Tue, 18 Oct 2011 21:14:12 GMT
Bok Tamara,

You didn't say what -Xmx value you are using.  Try a little higher value.  Note that loading
field values (and it looks like this one may be big because is compressed) from a lot of hits
is not recommended.

Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/


>________________________________
>From: Tamara Bobic <tamara.bobic@scai.fraunhofer.de>
>To: java-user@lucene.apache.org
>Cc: Roman Klinger <roman.klinger@scai.fraunhofer.de>
>Sent: Tuesday, October 18, 2011 12:21 PM
>Subject: OutOfMemoryError
>
>Hi all,
>
>I am using Lucene to query Medline abstracts and as a result I get around 3 million hits.
Each of the hits is processed and information from a certain field is used.
>
>After certain number of hits, somewhere around 1 million (not always the same number)
I get OutOfMemory exception that looks like this:
>
>Exception in thread "main" java.lang.OutOfMemoryError
>    at java.util.zip.Inflater.inflateBytes(Native Method)
>    at java.util.zip.Inflater.inflate(Inflater.java:221)
>    at java.util.zip.Inflater.inflate(Inflater.java:238)
>    at org.apache.lucene.document.CompressionTools.decompress(CompressionTools.java:108)
>    at org.apache.lucene.index.FieldsReader.uncompress(FieldsReader.java:609)
>    at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:385)
>    at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:231)
>    at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:1013)
>    at org.apache.lucene.index.DirectoryReader.document(DirectoryReader.java:520)
>    at org.apache.lucene.index.FilterIndexReader.document(FilterIndexReader.java:149)
>    at org.apache.lucene.index.IndexReader.document(IndexReader.java:947)
>    at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:152)
>    at org.apache.lucene.search.MultiSearcher.doc(MultiSearcher.java:156)
>    at org.apache.lucene.search.Hits.doc(Hits.java:180)
>    at de.fhg.scai.bio.tamara.corpusBuilding.LuceneCmdLineInterface.queryMedline(LuceneCmdLineInterface.java:178)
>    at de.fhg.scai.bio.tamara.corpusBuilding.LuceneCmdLineInterface.main(LuceneCmdLineInterface.java:152)
>
>
>this line which causes problems is:
>String docText = hits.doc(j).getField("DOCUMENT").stringValue() ; 
>
>I am using java 1.6 and I tried solving this issue with different garbage collectors (-XX:+UseParallelGC
and -XX:+UseParallelOldGC) but it didn't help.
>
>Does anyone have any idea how to solve this problem?
>
>There is also an official bug report:
>http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6293787
>
>Help is much appreciated. :)
>
>Best regards,
>Tamara Bobic
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message