lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautovic <emir.arnauto...@sematext.com>
Subject Re: Document Cache
Date Fri, 18 Mar 2016 15:56:06 GMT
Problem starts with autowarmCount="5000" - that executes 5000 queries 
when new searcher is created and as queries are executed, document cache 
is filled. If you have large queryResultWindowSize and queries return 
big number of documents, that will eat up memory before new search is 
executed. It probably takes some time as well.

This is also combined with filter cache. How big is your index?

Thanks,
Emir

On 18.03.2016 15:43, Rallavagu wrote:
> Thanks for the recommendations Shawn. Those are the lines I am 
> thinking as well. I am reviewing application also.
>
> Going with the note on cache invalidation for every two minutes due to 
> soft commit, wonder how would it go OOM in simply two minutes or is it 
> likely that a thread is holding the searcher due to long running query 
> that might be potentially causing OOM? Was trying to reproduce but 
> could not so far.
>
> Here is the filter cache config
>
> <filterCache class="solr.FastLRUCache" size="5000" initialSize="5000" 
> autowarmCount="1000"/>
>
> Query Results cache
>
> <queryResultCache class="solr.LRUCache" size="20000" 
> initialSize="20000" autowarmCount="5000"/>
>
> On 3/18/16 7:31 AM, Shawn Heisey wrote:
>> On 3/18/2016 8:22 AM, Rallavagu wrote:
>>> So, each soft commit would create a new searcher that would invalidate
>>> the old cache?
>>>
>>> Here is the configuration for Document Cache
>>>
>>> <documentCache class="solr.LRUCache" size="100000"
>>> initialSize="100000" autowarmCount="0"/>
>>>
>>> <enableLazyFieldLoading>true</enableLazyFieldLoading>
>>
>> In an earlier message, you indicated you're running into OOM.  I think
>> we can see why with this cache definition.
>>
>> There are exactly two ways to deal with OOM.  One is to increase the
>> heap size.  The other is to reduce the amount of memory that the program
>> requires by changing something -- that might be the code, the config, or
>> how you're using it.
>>
>> Start by reducing that cache size to 4096 or 1024.
>>
>> https://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap
>>
>> If yuo've also got a very large filterCache, reduce that size too.  The
>> filterCache typically eats up a LOT of memory, because each entry in the
>> cache is very large.
>>
>> Thanks,
>> Shawn
>>

-- 
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


Mime
View raw message