Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Date: Mon, 19 Dec 2016 10:47:15 -0700 (MST)
From: Chris Hostetter <hossman_lucene@fucit.org>
To: solr-user@lucene.apache.org
Subject: Re: Confusing debug=timing parameter
In-Reply-To: <CAGX0URid_crbhQRn3iAYVAzN-TVu=nTLDRX3yO6pcRyYXZ=fqw@mail.gmail.com>
Message-ID: <alpine.DEB.2.11.1612191026460.11265@tray>
References: <CAGX0URid_crbhQRn3iAYVAzN-TVu=nTLDRX3yO6pcRyYXZ=fqw@mail.gmail.com>
User-Agent: Alpine 2.11 (DEB 23 2013-08-11)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
archived-at: Mon, 19 Dec 2016 17:47:32 -0000


SG:

IIRC, when doing a distributed/cloud search, the timing info returned for 
each stage is the *cummulative* time spent on that stage in all shards -- 
so if you have 4 shards, the "process" time reported could be 4x as much 
as the actual process time spent.

The QTime reported back (in a distributed query) is the "wall clock' time 
spent by the single node that coordinated the response, *NOT* including 
time spent writting the response back to the client.


So let's imagine you have a single Solr node, and a client that says the 
total time for a Solr query took 5 seconds, but the QTime reported by solr 
is 1 second -- that suggests that 4 seconds was spent in either some sort 
of general network overhead between the client & solr, or in time spent 
reading docs from disk if (AND ONLY IF) Solr is running in single node 
mode, where Solr defers disk reads as long as possible and does those 
reads only as needed to write the response.  In this single node setup, 
you should *NEVER* see the cumulative debug=timing time exceed the QTime 
(for any reason i can think of)

In a distributed query, with multiple solr nodes, any discrepency between 
the QTime and the time reported by the client is going to be related to 
network overhead (or client overhead, parsing hte response) because Solr 
isn't able to do any "local reads" of docs from disk when writting the 
response.  imagine in this situation you have client reports 5 seconds, 
QTime reports 1 second, and debug=timing reports 45 seconds.   that 45 
seconds isn't "real" wall clock time, it's just 45 total seconds of time 
spent on all the nodes in parallel added up cumulatively.  the 5 second vs 
1 second discrepency would be from the network communication overhead, or 
client overhead in parsing hte response before proting "success"

does that make sense?


: Date: Sat, 17 Dec 2016 08:43:53 -0800
: From: S G <sg.online.email@gmail.com>
: Reply-To: solr-user@lucene.apache.org
: To: solr-user@lucene.apache.org
: Subject: Confusing debug=timing parameter
: 
: Hi,
: 
: I am using Solr 4.10 and its response time for the clients is not very good.
: Even though the Solr's plugin/stats shows less than 200 milliseconds,
: clients report several seconds in response time.
: 
: So I tried using debug-timing parameter from the Solr UI and this is what I
: got.
: Note how the QTime is 2978 while the time in debug-timing is 19320.
: 
: What does this mean?
: How can Solr return a result in 3 seconds when time taken between two
: points in the same path is 20 seconds ?
: 
: {
:   "responseHeader": {
:     "status": 0,
:     "QTime": 2978,
:     "params": {
:       "q": "*:*",
:       "debug": "timing",
:       "indent": "true",
:       "wt": "json",
:       "_": "1481992653008"
:     }
:   },
:   "response": {
:     "numFound": 1565135270,
:     "start": 0,
:     "maxScore": 1,
:     "docs": [
:       ....
:     ]
:   },
:   "debug": {
:     "timing": {
:       "time": 19320,
:       "prepare": {
:         "time": 4,
:         "query": {
:           "time": 3
:         },
:         "facet": {
:           "time": 0
:         },
:         "mlt": {
:           "time": 0
:         },
:         "highlight": {
:           "time": 0
:         },
:         "stats": {
:           "time": 0
:         },
:         "expand": {
:           "time": 0
:         },
:         "debug": {
:           "time": 0
:         }
:       },
:       "process": {
:         "time": 19315,
:         "query": {
:           "time": 19309
:         },
:         "facet": {
:           "time": 0
:         },
:         "mlt": {
:           "time": 1
:         },
:         "highlight": {
:           "time": 0
:         },
:         "stats": {
:           "time": 0
:         },
:         "expand": {
:           "time": 0
:         },
:         "debug": {
:           "time": 5
:         }
:       }
:     }
:   }
: }
: 

-Hoss
http://www.lucidworks.com/