lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: SolrCloud query logs, change from 4.10 to 5.5
Date Fri, 23 Sep 2016 15:29:18 GMT
Been there, done that....

You might be glad to know that there are a couple of tickets to reduce
the verbosity of the logs (or, more accurately, move some of the
logging to DEBUG level and allow a switch at startup) that should make
staring at logs less of a chore......

One other signal that a query is a sub-query to a shard is if
"distrib=false" is set but I don't know how reliable that would be for
your purpose...

Best,
Erick

On Fri, Sep 23, 2016 at 7:29 AM, Elaine Cario <etcario@gmail.com> wrote:
> Oh, never mind.  Apparently staring at logs has led to blindness...I do see
> the "master" query with the full elapsed time and hit count, and indeed,
> there is a  parameter "_" with some tracking number which links all the
> queries together.
>
>
> On Thu, Sep 22, 2016 at 7:32 PM, Elaine Cario <etcario@gmail.com> wrote:
>
>> We're in the process of upgrading from SolrCloud 4.10 to 5.5, and we
>> noticed a change in how distributed queries get logged.
>>
>> In Solr 4.10 we noted that the original node receiving the query logged
>> the query with a full hit count and elapsed time for the entire query,
>> using the original request handler (we don't use the default /select
>> handler).  The other nodes logged the queries sent out from the original
>> node, using the /select request handler.  These entries just included the
>> query stats from that particular node/shard.
>>
>> This made it easy when log-diving to differentiate between the stats for
>> the entire query completion, vs the individual stats for each shard, and we
>> were also able to detect any unexpected network latencies between the
>> shards.
>>
>> But now we are finding in Solr 5.5 that each shard just logs its own
>> stats, using the original request handler and there's no log entry for the
>> query as a whole.  This is making some of our existing log analysis
>> difficult when we try to tie it back to our other application logs.
>>
>> So, I have 2 questions:
>>
>> - is there a way to force a log entry for the complete query?
>> - is there some definitive way to link together all the log entries for a
>> query across the shards, e.g. some query parameter placed there by Solr?
>>  (In some cases our applications due add a custom param with a transaction
>> ID, but it's not consistent and I wonder if Solr is doing something or can
>> be configured to add something)
>>
>> Thanks.
>>

Mime
View raw message