lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: SolrCloud scaling/optimization for high request rate
Date Tue, 30 Oct 2018 02:56:58 GMT
Sofiya:

The interval between when a commit happens and all the autowarm
queries are finished if 52 seconds for the filterCache. seen warming
that that long unless something's very unusual. I'd actually be very
surprised if you're really only firing 64 autowarm queries and it's
taking almost 52 seconds.

However, if it really is taking that long for that few queries, then
most of what I think I know about this subject is probably wrong
anyway. I guess it could just be CPU starvation in that case. At any
rate, it's an anomaly that should get an explanation.

I suppose the other approach would be if you can get the autowarm time
when there's not much querying going on. If you get autowarm times of,
say, 40 seconds when there's no querying going on then it's not the
number of autowarm queries, but "something about your queries" that's
slowing things down, at least that gives you a place to start looking.

And I should have mentioned that when I think about excessive autowarm
counts, I've seen them in the thousands, 64 isn't really excessive,
even if diminishing returns start around 10-20...

Good Luck!
Erick

On Mon, Oct 29, 2018 at 10:54 AM Sofiya Strochyk <ss@interlogic.com.ua> wrote:
>
> Could you please clarify what is memory disk layer? Do you mean swapping from memory
to disk, reading from disk to memory, or something else?
>
>
> On 29.10.18 17:20, Deepak Goel wrote:
>
> I would then suspect performance is choking in memory disk layer. can you please check
the performance?
>
> On Mon, 29 Oct 2018, 20:30 Sofiya Strochyk, <ss@interlogic.com.ua> wrote:
>>
>> Hi Deepak and thanks for your reply,
>>
>>
>> On 27.10.18 10:35, Deepak Goel wrote:
>>
>>
>> Last, what is the nature of your request. Are the queries the same? Or they are very
random? Random queries would need more tuning than if the queries the same.
>>
>> The search term (q) is different for each query, and filter query terms (fq) are
repeated very often. (so we have very little cache hit ratio for query result cache, and very
high hit ratio for filter cache)
>>
>> --
>> Sofiia Strochyk
>>
>>
>>
>> ss@interlogic.com.ua
>>
>> www.interlogic.com.ua
>>
>>
>
>
> --
> Sofiia Strochyk
>
>
>
> ss@interlogic.com.ua
>
> www.interlogic.com.ua
>
>

Mime
View raw message