lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: maxwarmingSearchers and memory leak
Date Fri, 24 Feb 2017 14:17:24 GMT
On 2/23/2017 1:51 AM, SOLR4189 wrote:
> We have maxwarmingSearchers set to 2 and field value cache set to
> initial size of 64. We saw that by taking a heap dump that our caches
> consume 70% of the heap size, by looking into the dump we saw that
> fieldValueCache has 6 occurences of
> org.apache.solr.util.concurrentCache. When we have
> maxWarmingSearches=2 we would expect to have only 3 (maybe 4 before GC
> has been launched). What can it be? We use solr4.10.1 

There are no specific details here about your setup except for
maxWarmingSearchers (which is at the typical default value) and
fieldValueCache, which is not explicitly configured in any example, and
probably should not be explicitly configured.  The version number is not
provided.  Heap size is not provided.  Installation method is not
provided.  Other details about your OS, Solr install, configuration, and
index are not available.

Version 4.10.x can be installed in a variety of ways, most of which are
outside the project's control.  5.0 and later are typically only
installed using the built-in container (jetty) and scripts.

Here's a starting list for additional info that we may need:

* OS vendor and version
* Java vendor and version
* Servlet container used to start Solr.
* Total amount of memory in the server.
* Max heap size for Solr.
* An idea of exactly what is running on the server.
* Total index size and document count being handled by Solr (add up all
indexes).
* A screen shot of a process list sorted by memory usage.
* A screenshot showing total system memory allocations.

Those last two are typically available with one screen on most operating
systems that have the "top" utility, if it allows sorting by memory by
pressing shift-M.

Although a memory leak is certainly possible, I am not aware of any
known problems with memory leaks in 4.10 or any of the current code
branches.  Very likely you simply need a larger heap for Solr to do the
work that it has been asked to do.  This link may be helpful:

https://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap

There is no "ConcurrentCache" class in Solr.  There is
ConcurrentLRUCache, though.  I have no idea how large cache entries are
in the fieldValueCache, though I would expect them to be fairly small. 
One cache that typically has very large entry sizes is filterCache. 
Each entry in that cache has a byte size equal to the maxDoc value of
the index divided by eight.  So an index with one million documents in
it has filterCache entries which are each 125000 bytes.  An index with
100 million documents has filterCache entries which are each 12.5
million bytes.

Thanks,
Shawn


Mime
View raw message