Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 1205 invoked from network); 12 Jul 2007 23:12:59 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 12 Jul 2007 23:12:59 -0000 Received: (qmail 56413 invoked by uid 500); 12 Jul 2007 23:12:55 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 55880 invoked by uid 500); 12 Jul 2007 23:12:54 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 55869 invoked by uid 99); 12 Jul 2007 23:12:54 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jul 2007 16:12:54 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (herse.apache.org: domain of DORONC@il.ibm.com designates 195.212.29.153 as permitted sender) Received: from [195.212.29.153] (HELO mtagate4.de.ibm.com) (195.212.29.153) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jul 2007 16:12:51 -0700 Received: from d12nrmr1607.megacenter.de.ibm.com (d12nrmr1607.megacenter.de.ibm.com [9.149.167.49]) by mtagate4.de.ibm.com (8.13.8/8.13.8) with ESMTP id l6CNCTXp053860 for ; Thu, 12 Jul 2007 23:12:29 GMT Received: from d12av04.megacenter.de.ibm.com (d12av04.megacenter.de.ibm.com [9.149.165.229]) by d12nrmr1607.megacenter.de.ibm.com (8.13.8/8.13.8/NCO v8.3) with ESMTP id l6CNBEWI2240542 for ; Fri, 13 Jul 2007 01:11:14 +0200 Received: from d12av04.megacenter.de.ibm.com (loopback [127.0.0.1]) by d12av04.megacenter.de.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id l6CNBEmV007224 for ; Fri, 13 Jul 2007 01:11:14 +0200 Received: from d12mc102.megacenter.de.ibm.com (d12mc102.megacenter.de.ibm.com [9.149.167.114]) by d12av04.megacenter.de.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id l6CNBEAi007221 for ; Fri, 13 Jul 2007 01:11:14 +0200 In-Reply-To: <11510351.post@talk.nabble.com> Subject: Re: Lucene RAM Directory doesn't work for Index Size > 8 GB To: java-user@lucene.apache.org X-Mailer: Lotus Notes Release 7.0 HF277 June 21, 2006 Message-ID: From: Doron Cohen Date: Thu, 12 Jul 2007 16:08:47 -0700 X-MIMETrack: Serialize by Router on D12MC102/12/M/IBM(Release 7.0.2HF71 | November 3, 2006) at 13/07/2007 02:11:13 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII X-Virus-Checked: Checked by ClamAV on apache.org hi Murali, I found a casting issue that can cause this problem - see patch in http://issues.apache.org/jira/browse/LUCENE-957 is the problem solved with this patch? Doron muraalee wrote on 09/07/2007 14:15:04: > > Hi, > > We are facing a strange problem with RAMDirectory for indices > greater than 8 > GB. We have indexed around 6.5 million lucene documents and theindex size > is around 8 GB. Below is the contents of Index Directory. > > 2236964197 _1x.fdt > 51811488 _1x.fdx > 293 _1x.fnm > 2234929832 _1x.frq > 369156856 _1x.nrm > 2746607717 _1x.prx > 4979893 _1x.tii > 389652682 _1x.tis > 42 segments_3x > 20 segments.gen > > We could open this using FSDirectory and it seems to be working > fine. But if > we switch to RAMDirectory, we get the following exception. > > - We are using Lucene 2.2.0 currently > > Caused by: java.lang.ArrayIndexOutOfBoundsException: -369 > at > org.apache.lucene.store.RAMInputStream.readByte(RAMInputStream.java:64) > at org.apache.lucene.store.IndexInput. > readVLong(IndexInput.java:77) > at > org.apache.lucene.index.MultiLevelSkipListReader. > loadSkipLevels(MultiLevelSkipListReader.java:194) > at > org.apache.lucene.index.MultiLevelSkipListReader. > skipTo(MultiLevelSkipListReader.java:97) > at > org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDocs. > java:164) > at > org.apache.lucene.search.PhrasePositions. > skipTo(PhrasePositions.java:52) > at > org.apache.lucene.search.PhraseScorer.doNext(PhraseScorer.java:88) > at org.apache.lucene.search.PhraseScorer. > next(PhraseScorer.java:81) > at > org.apache.lucene.search.DisjunctionSumScorer. > initScorerDocQueue(DisjunctionSumScorer.java:105) > at > org.apache.lucene.search.DisjunctionSumScorer. > next(DisjunctionSumScorer.java:144) > at > org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2. > java:327) > at > org.apache.lucene.search.IndexSearcher.search(IndexSearcher. > java:146) > at org.apache.lucene.search.Searcher.search(Searcher. > java:118) > at org.apache.lucene.search.Searcher.search(Searcher.java:97) > at > org.apache.lucene.search.QueryWrapperFilter. > bits(QueryWrapperFilter.java:50) > > > Any help to address this problem is greatly appreciated. > > > thanks > Murali V --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org