lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chandan Tamrakar" <chan...@ccnep.com.np>
Subject RE: Searching with too many clauses + Out of Memory
Date Wed, 01 Aug 2007 15:09:17 GMT
What is the size of heap u r allocating for your app ?

-----Original Message-----
From: Harini Raghavan [mailto:harini.raghavan@insideview.com] 
Sent: Wednesday, August 01, 2007 2:29 PM
To: java-user@lucene.apache.org
Subject: Searching with too many clauses + Out of Memory

 Hi Everyone,

I am using Compass 1.1 M2 which supports Lucene 2.2 to store & search huge
amount of company, executive and employment data. There are some usecases
where I need to search for executives/employments on the result set of
company search. But when I try to create a compass query to search for
executives for over 1 lac company ids, it runs out of memory as the query is
huge. Here is the exception stack trace:

java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:342)
at org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:435)
at org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:428)
at org.apache.lucene.index.MultiTermDocs.read(MultiReader.java:393)
at org.apache.lucene.search.TermScorer.next(TermScorer.java:106)
at org.apache.lucene.util.ScorerDocQueue.topNextAndAdjustElsePop(
ScorerDocQueue.ja va:116)
at
org.apache.lucene.search.DisjunctionSumScorer.advanceAfterCurrent(Disjunctio
nSu
mScorer.java:175)
at org.apache.lucene.search.DisjunctionSumScorer.next(
DisjunctionSumScorer.java:14 6)
at org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:327)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:124)
at org.apache.lucene.search.MultiSearcher.search(MultiSearcher.java:232)
at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:74)
at org.apache.lucene.search.Hits.<init>(Hits.java:61)
at org.apache.lucene.search.Searcher.search(Searcher.java:55)
at
org.compass.core.lucene.engine.transaction.ReadCommittedTransaction.findByQu
ery(
ReadCommittedTransaction.java:469)
at
org.compass.core.lucene.engine.transaction.ReadCommittedTransaction.doFind(R
ead
CommittedTransaction.java:426)
at
org.compass.core.lucene.engine.transaction.AbstractTransaction.find(Abstract
Tra
nsaction.java:91)
at org.compass.core.lucene.engine.LuceneSearchEngine.find(
LuceneSearchEngine.java: 379)
at
org.compass.core.lucene.engine.LuceneSearchEngineQuery.hits(LuceneSearchEngi
neQ
uery.java:151)
at
org.compass.core.impl.DefaultCompassQuery.hits(DefaultCompassQuery.java:133)

at
org.compass.core.support.search.CompassSearchHelper.performSearch(CompassSea
rch
Helper.java:144)
at
org.compass.core.support.search.CompassSearchHelper$1.doInCompass(CompassSea
rch
Helper.java:89)
at org.compass.core.CompassTemplate.execute(CompassTemplate.java:137)
at
org.compass.core.support.search.CompassSearchHelper.search(CompassSearchHelp
er.
java:86)

It looks like this error is actually in the lucene code. It would be great
if anyone in this group has an idea about this kind of usecase and has some
suggestions.

Thanks,
Harini



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message