lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Memory usage
Date Thu, 30 Sep 2010 03:07:34 GMT
How many documents are there? How many unique words are in a text
field? Both of these numbers can have a non-linear effect on the
amount of space used.

But, usually a 22Gb index (on disk) might need 6-12G of ram total.
There is something odd going on here.

Lance

On Wed, Sep 29, 2010 at 4:34 PM, Jeff Moss <jmoss@heavyobjects.com> wrote:
> My server has 128GB of ram, the index is 22GB large. It seems the memory
> consumption goes up on every query and the garbage collector will never free
> up as much memory as I expect it to. The memory consumption looks like a
> curve, it eventually levels off but the old gen is always 60 or 70GB. I have
> tried adjusting the cache settings but it doesn't seem to make any
> difference.
>
> Is there something I'm doing wrong or is this expected behavior?
>
> Here is a screenshot of what I see in jconsole after running for a few
> minutes:
> http://i51.tinypic.com/2qntca1.png
>
> Here is a 24 hour period of the same data taken from a custom jmx monitor:
> http://i51.tinypic.com/2vcu9u8.png
>
> The server performs pretty much as good at the beginning of this cycle as it
> does at the end so all of this memory accumulation seems to not be doing
> anything useful.
>
> I am running the 1.4 war but I was having this problem with 1.3 also. Tomcat
> 6.0.18, Java 1.6.0. I haven't gone as far as doing any memory profiling or
> java debugging because I'm inexperienced, but that will be the next thing I
> try. Any help would be appreciated.
>
> Thanks,
>
> -Jeff
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message