lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Liew <adrian.l...@avanade.com>
Subject RE: Solr Cloud 5.3.0 Errors in Logs
Date Tue, 17 Nov 2015 03:11:30 GMT
Thanks for the tip Eric. Really useful article to know.

I will keep you posted on my findings!

Regards,
Adrian

-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Tuesday, November 17, 2015 8:56 AM
To: solr-user <solr-user@lucene.apache.org>
Subject: Re: Solr Cloud 5.3.0 Errors in Logs

Here's perhaps more than you really want to know about commits....

https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

The short form is that most setups set autoCommit to a relatively short interval (15-60 seconds
at least under heavy indexing loads, I've seen 10-15 minutes under relatively light loads)
with openSearcher set to false. And I rarely set maxDocs in that configuration, it's actually
not that useful IMO.

Then set up an autoSoftCommit to be as long as you can tolerate, but IMO rarely shorter than
60 seconds unless you have very aggressive near-real-time (NRT) requirements.

And if your product manager simply insists on very aggressive NRT settings, you should consider
making your filterCache and queryResultCache relatively small with minimal autowarming.

Any time you exceed maxWarmingSearchers, it indicates poorly configured Solr instances and/or
commits happening far too often.
Bumping that up to numbers greater than two is almost always a band-aid over that misconfiguration.

On Mon, Nov 16, 2015 at 3:45 PM, Adrian Liew <adrian.liew@avanade.com> wrote:
> Thanks Eric.
>
> Here is my reply
>
>>> 1> that commits are happening too often, and especially if the 
>>> 1> commits
>>> are happening
>>> from a client. If commits aren't being sent by a client, then look at autoCommit
and softAutoCommit in solrconfig.xml (if you can).
> Understand what you mean. Besides talking to to the folks at Sitecore on where they issue
commits, is there a way I can balance these with autoCommit and softAutoCommit in solrconfig.xml?
Best is, can you recommend any articles that talk about best practice configuration for a
production setup.
>
>>> 2> excessive autowarm settings, again in solronfig.xml.
>>> If, as you say all of Solr is a black box, then talk to the Sitecore folks, on
the surface Solr is just poorly configured.
> I will raise with Sitecore guys on this. Particularly asking them why commits are happening
faster than searchers can be opened. I have seen overlappingDeckSearchers have exceeded the
limit errors as well. Let you know what I get back from the guys.
>
> Regards,
> Adrian
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Tuesday, November 17, 2015 12:44 AM
> To: solr-user <solr-user@lucene.apache.org>
> Subject: Re: Solr Cloud 5.3.0 Errors in Logs
>
> Having 6 warming serachers is an anti-pattern. What it means is that commits are happening
faster than your searcher can be opened. There is _no_ good reason that I know of for changing
it from 2, having changed it in solrconfig.xml to 6 almost always indicates an improper configuration.
>
> Places to look:
> 1> that commits are happening too often, and especially if the commits
> are happening
> from a client. If commits aren't being sent by a client, then look at autoCommit and
softAutoCommit in solrconfig.xml (if you can).
>
> 2> excessive autowarm settings, again in solronfig.xml.
>
> If, as you say all of Solr is a black box, then talk to the Sitecore folks, on the surface
Solr is just poorly configured.
>
> Best,
> Erick
>
> On Mon, Nov 16, 2015 at 4:33 AM, Adrian Liew <adrian.liew@avanade.com> wrote:
>> Hi Emir,
>>
>> I am working with a third party platform, Sitecore. The product is a black box that
encapsulates the internal workings of solr queries and so on. If there are any questions you
have with regards with the below, let me know. It will be useful for me to communicate what
could cause the issues below.
>>
>> Regards,
>> Adrian
>>
>> -----Original Message-----
>> From: Emir Arnautovic [mailto:emir.arnautovic@sematext.com]
>> Sent: Monday, November 16, 2015 4:47 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Solr Cloud 5.3.0 Errors in Logs
>>
>> Hi Adrian,
>> Can you give us bit more details about warmup queries you use and test that you are
running when error occurs.
>>
>> Thanks,
>> Emir
>>
>> On 16.11.2015 08:40, Adrian Liew wrote:
>>> Hi there,
>>>
>>> Will like to get some opinions on the errors encountered below. I have currently
setup a SolrCloud cluster of 3 servers (each server hosting a Solr instance and a Zookeeper
instance).
>>>
>>> I am encountering the errors below in the logs:
>>> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException:
Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later.
>>> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException:
Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later.
>>> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException:
Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later.
>>> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCore org.apache.solr.common.SolrException:
Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later.
>>> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1:
Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later.
>>> Monday, November 16, 2015 3:22:54 PM ERROR null SolrCmdDistributor org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica1:
Error opening new searcher. exceeded limit of maxWarmingSearchers=6, try again later.
>>> Monday, November 16, 2015 3:22:54 PM WARN null 
>>> DistributedUpdateProcessor Error sending update to 
>>> http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM 
>>> WARN null DistributedUpdateProcessor Error sending update to 
>>> http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM 
>>> WARN null DistributedUpdateProcessor Error sending update to 
>>> http://172.18.111.112:8983/solr Monday, November 16, 2015 3:22:54 PM 
>>> WARN null DistributedUpdateProcessor Error sending update to 
>>> http://172.18.111.112:8983/solr
>>>
>>> 11/16/2015, 3:17:09 PM
>>>
>>> WARN
>>>
>>> null
>>>
>>> DistributedUpdateProcessor
>>>
>>> Error sending update to http://172.18.111.112:8983/solr
>>>
>>> 11/16/2015, 3:17:09 PM
>>>
>>> WARN
>>>
>>> null
>>>
>>> DistributedUpdateProcessor
>>>
>>> Error sending update to http://172.18.111.112:8983/solr
>>>
>>> 11/16/2015, 3:22:26 PM
>>>
>>> ERROR
>>>
>>> null
>>>
>>> SolrCmdDistributor
>>>
>>> org.apache.solr.client.solrj.SolrServerException: Timeout occured 
>>> while waiting response from server at:
>>> http://172.18.111.112:8983/solr/sitecore_master_index_shard1_replica
>>> 1
>>>
>>>
>>>
>>> Main errors are Timeout occurred exceptions, maxWarmingSearchers exceeded. Is
anyone able to advise or have experienced something the same as the above in their SolrCloud
setup?
>>>
>>> Regards,
>>> Adrian
>>>
>>>
>>>
>>
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log 
>> Management Solr & Elasticsearch Support * http://sematext.com/
>>
Mime
View raw message