lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: OOM during IndexReader open
Date Sat, 02 Jun 2012 16:02:47 GMT
Have you tried a System.gc before opening the index? And check how much 
memory is available before the open. Does the open seem to be taking more 
memory that could reasonably be expected? How big is the index, how many 
docs, how many unique terms in that index?

1.8G doesn't sound like a lot. You may simply have to bite the bullet and 
move to an environment with more memory. Or, split the index and run on 
multiple machines/JVMs.

Have many deletes been performed? How many segments are there? Maybe you can 
merge using a standalone program that doesn't have the rest of your app 
consuming memory.

-- Jack Krupansky

-----Original Message----- 
From: nishesh.gupta@emc.com
Sent: Friday, June 01, 2012 7:53 PM
To: java-user@lucene.apache.org
Subject: OOM during IndexReader open

Hi,

I am getting the following OOM consistently whenever the index is opened . 
Is it because now the index is holding too many terms ? Our application ( 
that has Lucene 2.9.3 ) already has reached 1.8G of virtual memory.
Is there a way I can avoid OOM during IndexReader open because of too many 
terms ?


2012-05-30 14:12:46,139 [MC:10.2.2.1-1332015615933-1264 
FS:parinfprdcl01k_655394_0] ERROR util  - Context: MC: 10.2.2 
.1-1332015615933-1264 failed at the state LocalIndexOptimizeAndCompress, 
NumActiveThreads = 64. Error: java.lang.OutO        fMemoryError: Java heap 
space % STACK:
org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:90)
org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:122)
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:652)
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:613)
org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:106)
org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:71)
org.apache.lucene.index.IndexReader.open(IndexReader.java:482)
org.apache.lucene.index.IndexReader.open(IndexReader.java:320)
com.kazeon.search.indexingengine.context.MergerContext.deleteAgainstFiler(MergerContext.java:1357)
com.kazeon.search.indexingengine.context.MergerContext.deleteAgainstFiler(MergerContext.java:656)
com.kazeon.search.indexingengine.statemachine.modifiers.merger.LocalIndexOptimizeAndCompressModifier.modifyState

Aux(LocalIndexOptimizeAndCompressModifier.java:303)
com.kazeon.search.indexingengine.statemachine.modifiers.merger.LocalIndexOptimizeAndCompressModifier.mergeAllICs

(LocalIndexOptimizeAndCompressModifier.java:175)
com.kazeon.search.indexingengine.statemachine.modifiers.merger.LocalIndexOptimizeAndCompressModifier.modifyState

(LocalIndexOptimizeAndCompressModifier.java:106)
com.kazeon.util.scoreboard.WorkerThread.run(WorkerThread.java:31)


thanks,
Nishesh 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message