lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Should heap size be proportionate to the size of the index I'm opening?
Date Mon, 11 Mar 2013 09:18:20 GMT
It's not that simple.   More to do with number of terms than raw index
size.  Of course your large index may well have more terms than a
smaller one.

See  http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html
and http://searchhub.org/2011/09/14/estimating-memory-and-storage-for-lucenesolr/

They'll be about versions more recent than 3.0.3 but the underlying
concepts may still be applicable.

512Mb for a 70Gb index sounds very conservative.



--
Ian.


On Mon, Mar 11, 2013 at 9:08 AM, Gili Nachum <gilinachum@gmail.com> wrote:
> Hello.
>
> I'm getting an OOME with a heap size of 512MB while trying to open an
> IndexReader over a 70GB index (FSDirectory).
> It didn't happen before when the index was smaller, and it doesn't happen
> if Increase the heap size above 512MB.
>
> Does this mean that the required heap size is always proportionate to the
> size of the index?
>
> Linux, Lucene 3.0.3 (yeah it's old...).

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message