lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Santamauro <david.santama...@gmail.com>
Subject Re: collection mbeans: requests
Date Tue, 04 Aug 2015 18:06:47 GMT

I have your suggested shards.qt set up in another collection for another 
reason but I'll do that redirect here as well, thanks for the confirmation.

On 08/04/2015 10:45 AM, Shawn Heisey wrote:
> On 8/4/2015 5:19 AM, David Santamauro wrote:
>>
>> I have a question about how the stat 'requests' is calculated. I would
>> really appreciate it if anyone could shed some light on the figures below.
>>
>> Assumptions:
>>    version: 5.2.0
>>    layout: 8 node solrcloud, no replicas (node71-node78)
>>    collection: col1
>>    handler: /search
>>    stats request: /col1/admin/mbeans?stats=true&cat=QUERYHANDLER&wt=json'
>>
>> I wrote a simple shell script that grabs the requests stats member from
>> every node.
>>
>> After collection reload
>> node 71 -- requests: 2
>> node 72 -- requests: 2
>> node 73 -- requests: 2
>> node 74 -- requests: 2
>> node 75 -- requests: 2
>> node 76 -- requests: 2
>> node 77 -- requests: 2
>> node 78 -- requests: 2
>> * I assume these are the auto-warm searches
>>
>>
>> After submitting 1 request (q=*:*)
>> node 71 -- requests: 4
>> node 72 -- requests: 3
>> node 73 -- requests: 3
>> node 74 -- requests: 3
>> node 75 -- requests: 3
>> node 76 -- requests: 4
>> node 77 -- requests: 3
>> node 78 -- requests: 3
>>
>> After resubmitting the same request
>> node 71 -- requests: 6
>> node 72 -- requests: 4
>> node 73 -- requests: 4
>> node 74 -- requests: 4
>> node 75 -- requests: 4
>> node 76 -- requests: 5
>> node 77 -- requests: 5
>> node 78 -- requests: 4
>>
>> If that wasn't strange enough, things get out of control if I add in
>> facet.pivot parameter(s)
>>
>> Fresh after reload (see above, 2 for every node)
>>
>> Total after a facet.pivot on two fields
>> node 71 -- requests: 13
>> node 72 -- requests: 15
>> node 73 -- requests: 14
>> node 74 -- requests: 12
>> node 75 -- requests: 14
>> node 76 -- requests: 12
>> node 77 -- requests: 14
>> node 78 -- requests: 12
>>
>> I imagine I'm seeing the internal cross-talk between nodes and if so,
>> how can one reliably keep stats on the number of "real" requests?
>
> Queries on distributed indexes change from the one request that you make
> into a request to every shard, to check for relevant documents.  If
> relevant documents are found, a second call to those specific shards is
> made to retrieve those documents.  So if you have 5 shards in your
> index, there could be up to 11 requests counted for a single query.  If
> all the shards are on separate nodes, then for that 11-request query,
> one of those nodes would count three requests and the others would count
> two.
>
> I know what I'm going to say next would work on an index that is
> distributed but *not* SolrCloud, and I think it will work in SolrCloud too.
>
> If you add a "shards.qt" parameter to defaults in your main request
> handler (usually /select) that points at another, identically configured
> handler (perhaps named "/shards") that is also in solrconfig.xml, then
> that other handler should receive the distributed requests and the main
> handler should only count the "real" requests.  You would be able to
> track those numbers separately.
>
> Thanks,
> Shawn
>

Mime
View raw message