lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aleksander M. Stensby" <aleksander.sten...@integrasco.no>
Subject Re: how to estimate how much memory is required to support the large index search
Date Mon, 17 Nov 2008 08:31:04 GMT
One major factor that may result in heap space problems is if you are  
doing any form of sorting when searching. Do you have any form of default  
sort in your application? Also, the type of field used for sorting is  
important with regard to memory consumption.

This issue has been discussed before on the list. (You can search the  
archive for sorting and memory consumption.)

- Aleksander

On Sun, 16 Nov 2008 14:36:36 +0100, Zhibin Mai <zbmai@yahoo.com> wrote:

> Hello,
>
> I
> am a beginner on using lucene. We developed an application to
> create and search index using lucene 2.3.1. We would like to know how
> to estimate how much memory is required to support
> the index search given an index.
>
> Recently,
> the size of the index has reached to about 200GB with 197M of documents
> and 223M of terms. Our application starts having intermittent
> "OutOfMemoryError: Java heap space" when we use
> it to search the index. We use JProfiler to get the following memory  
> allocation when we do one keyword search:
>
> char[]                                                        332MB
> org.apache.lucene.index.TermInfo            194MB
> java.lang.String                                        146MB
> org.apache.lucene.index.Term                99,823KB
> org.apache.lucene.index.Term                24,956KB
> org.apache.lucene.index.TermInfo[]        24,956KB
>
> byte[]                                                    188MB
> long[]                                                    49,912KB
>
> The memory allocation for the first 6 types of objects does not change  
> when we change the search criteria. Could you please give me some advice  
> what major factors will affect the memory allocation
> and how those factors will affect the memory usage precisely on search?  
> Is it possible to reduce the memory usage on search?
>
>
> Thank you,
>
>
> Zhibin
>
>
>



-- 
Aleksander M. Stensby
Senior software developer
Integrasco A/S
www.integrasco.no

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message