lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: lucene norms cached twice
Date Fri, 29 Oct 2010 17:27:14 GMT
Norms should not normally be loaded twice.

Since 2.9, searching is done at the sub-reader level, and so norms
should never be loaded for the main reader.

But can you describe how you're using Lucene?

Mike

On Fri, Oct 29, 2010 at 9:27 AM, Cabansag, Ronald-Alvin R
<ronald-alvin.cabansag@cengage.com> wrote:
>
> We are working with a large readonly lucene index(single segment) with large number of
fields and documents and are running into memory usage problems.
>
> We found that when using a ReadOnlyDirectoryReader and IndexSearcher created using the
same reader, the norms are cached twice - first by the reader itself and second by the reader's
subreaders. Is there an easy way to avoid having the norms cached twice when we only have
a single subreader?
>
> We thought of the following options:
> 1.) pass in the main reader as a subreader when creating the  IndexSearcher?  ( e.g.
new IndexSearcher(mainReader, IndexReader[] {mainReader}, int[] {0} )
> 2.) override ReadOnlyDirectoryReader.getSequentialSubReaders() method and return null.
This tells the IndexSearcher to use the main reader- ReadOnlyDirectoryReader.
> 3.) use SegmentReader.get(boolean, SegmentInfo, int) to create a ReadOnlySegmentReader
that we use as our main reader instead.
>
> Are there any negative implications to the above approaches? Or are there better approaches
to the problem?
>
> Thanks in advance for any help.
>
> Alvin Cab
> Cengage Learning
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message