lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gili Nachum <gilinac...@gmail.com>
Subject Re: Should heap size be proportionate to the size of the index I'm opening?
Date Mon, 11 Mar 2013 15:57:07 GMT
Great links. Thanks Ian.
Good to know that Lucene v4, has a smaller heap foot print.


On Mon, Mar 11, 2013 at 11:18 AM, Ian Lea <ian.lea@gmail.com> wrote:

> It's not that simple.   More to do with number of terms than raw index
> size.  Of course your large index may well have more terms than a
> smaller one.
>
> See
> http://blog.mikemccandless.com/2010/07/lucenes-ram-usage-for-searching.html
> and
> http://searchhub.org/2011/09/14/estimating-memory-and-storage-for-lucenesolr/
>
> They'll be about versions more recent than 3.0.3 but the underlying
> concepts may still be applicable.
>
> 512Mb for a 70Gb index sounds very conservative.
>
>
>
> --
> Ian.
>
>
> On Mon, Mar 11, 2013 at 9:08 AM, Gili Nachum <gilinachum@gmail.com> wrote:
> > Hello.
> >
> > I'm getting an OOME with a heap size of 512MB while trying to open an
> > IndexReader over a 70GB index (FSDirectory).
> > It didn't happen before when the index was smaller, and it doesn't happen
> > if Increase the heap size above 512MB.
> >
> > Does this mean that the required heap size is always proportionate to the
> > size of the index?
> >
> > Linux, Lucene 3.0.3 (yeah it's old...).
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message