lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From muraalee <muraa...@gmail.com>
Subject Lucene RAM Directory doesn't work for Index Size > 8 GB
Date Mon, 09 Jul 2007 21:15:04 GMT

Hi,

We are facing a strange problem with RAMDirectory for indices greater than 8
GB. We have indexed around 6.5 million lucene documents and the index size
is around 8 GB. Below is the contents of Index Directory.

2236964197  _1x.fdt
   51811488  _1x.fdx
           293  _1x.fnm
2234929832  _1x.frq
 369156856  _1x.nrm
2746607717  _1x.prx
     4979893  _1x.tii
 389652682  _1x.tis
            42  segments_3x
            20  segments.gen

We could open this using FSDirectory and it seems to be working fine. But if
we switch to RAMDirectory, we get the following exception.

- We are using Lucene 2.2.0 currently 

Caused by: java.lang.ArrayIndexOutOfBoundsException: -369      
        at
org.apache.lucene.store.RAMInputStream.readByte(RAMInputStream.java:64)
        at org.apache.lucene.store.IndexInput.readVLong(IndexInput.java:77)      
        at
org.apache.lucene.index.MultiLevelSkipListReader.loadSkipLevels(MultiLevelSkipListReader.java:194)
   
        at
org.apache.lucene.index.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:97)
     
        at
org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDocs.java:164)                
        at
org.apache.lucene.search.PhrasePositions.skipTo(PhrasePositions.java:52)      
        at
org.apache.lucene.search.PhraseScorer.doNext(PhraseScorer.java:88)      
        at org.apache.lucene.search.PhraseScorer.next(PhraseScorer.java:81)      
        at
org.apache.lucene.search.DisjunctionSumScorer.initScorerDocQueue(DisjunctionSumScorer.java:105)
 
        at
org.apache.lucene.search.DisjunctionSumScorer.next(DisjunctionSumScorer.java:144)    
        at
org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:327)          
        at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)        
        at org.apache.lucene.search.Searcher.search(Searcher.java:118)        
        at org.apache.lucene.search.Searcher.search(Searcher.java:97)    
        at
org.apache.lucene.search.QueryWrapperFilter.bits(QueryWrapperFilter.java:50)  


Any help to address this problem is greatly appreciated.


thanks
Murali V 
-- 
View this message in context: http://www.nabble.com/Lucene-RAM-Directory-doesn%27t-work-for-Index-Size-%3E-8-GB-tf4052298.html#a11510351
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message