Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 17387 invoked from network); 29 Oct 2010 13:27:40 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 29 Oct 2010 13:27:40 -0000 Received: (qmail 43713 invoked by uid 500); 29 Oct 2010 13:27:37 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 43495 invoked by uid 500); 29 Oct 2010 13:27:37 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 43486 invoked by uid 99); 29 Oct 2010 13:27:36 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Oct 2010 13:27:36 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [69.32.146.53] (HELO thomsonlearning.com) (69.32.146.53) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Oct 2010 13:27:30 +0000 Received: from ([10.160.3.239]) by ohciniron02.thomsonlearning.com with ESMTP with TLS id 5502562.76062455; Fri, 29 Oct 2010 09:27:07 -0400 Received: from OHCINMBX01.corp.local ([10.160.3.160]) by ohcinht01.corp.local ([10.160.3.239]) with mapi; Fri, 29 Oct 2010 09:27:07 -0400 From: "Cabansag, Ronald-Alvin R" To: "java-user@lucene.apache.org" Date: Fri, 29 Oct 2010 09:27:06 -0400 Subject: lucene norms cached twice Thread-Topic: lucene norms cached twice Thread-Index: Act26B5/adXQQUeRTHuugch/PftYNQAhLg9g Message-ID: <09BA120FB4BE134095E63B761C194E8026C8F461F0@OHCINMBX01.corp.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 We are working with a large readonly lucene index(single segment) with larg= e number of fields and documents and are running into memory usage problems= . We found that when using a ReadOnlyDirectoryReader and IndexSearcher create= d using the same reader, the norms are cached twice - first by the reader i= tself and second by the reader's subreaders. Is there an easy way to avoid = having the norms cached twice when we only have a single subreader? We thought of the following options: 1.) pass in the main reader as a subreader when creating the IndexSearcher= ? ( e.g. new IndexSearcher(mainReader, IndexReader[] {mainReader}, int[] {= 0} ) 2.) override ReadOnlyDirectoryReader.getSequentialSubReaders() method and r= eturn null. This tells the IndexSearcher to use the main reader- ReadOnlyDi= rectoryReader. 3.) use SegmentReader.get(boolean, SegmentInfo, int) to create a ReadOnlySe= gmentReader that we use as our main reader instead. Are there any negative implications to the above approaches? Or are there b= etter approaches to the problem? Thanks in advance for any help. Alvin Cab Cengage Learning --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org