lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Memory Using In Faceted Search (UnInvertedField's)
Date Wed, 18 Sep 2013 21:04:02 GMT
On 9/18/2013 11:08 AM, anton@swooptalent.com wrote:
> I'm using Solr 4.3.1 for faceted search and have 4 fields used for faceting. My question
is about memory consumtion.
> I've set up heap size to use 6Gb of RAM, but I see in resource monitor it uses much more
than that - up to 10Gb where 4 Gb is reported as shareable memory.
> I've calculated the size of cached set of UnInverted fields and it's 2Gb - I'm fine with
that, both GC monitor and 'fieldValueCache' stats in 'Plugins/Stats' UI for Solr report that.
But I can't understand what's that memory that's being reserved after filling in fieldValueCache
with uninverted fields (right in UnInvertedField.uninvert method) and not used  (or not released).
> Is that some memory leak? Or is that something I should tune with garbage collector by
making it more aggressive (GC only shows me 2.x Gb in Old Space and I see those UnInvertedField's
there in heap dump)?

I have noticed the same thing.  I do not think there is an actual
problem, but just something strange with the operating system memory
reporting.

https://www.dropbox.com/s/zacp4n3gu8wb9ab/idxb1-top-sorted-mem.png

In the screenshot above, you can see that there is 64GiB total memory.
There is 44449012k being used by the OS disk cache and 9853824k free
memory.  If you add these two numbers up, you get a number that's
roughly 51 GiB (54302836k).

You can also see that it says Solr (4.2.1) has a resident size of 16g,
with 11g of that in shareable memory.  FYI, the max java heap is 6g,
verified by the Solr dashboard and tools like jconsole.

With these numbers, if Solr really did have a memory resident size of
16g, Solr's memory size plus the combined total of cached and free
memory would require 3g of swap, but as you can see, there is zero swap
in use.

I don't know if the reporting problem can be fixed.  It is interesting
to know that the same thing happens on both Linux and Windows.

Thanks,
Shawn


Mime
View raw message