lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1972) Need additional query stats in admin interface - median, 95th and 99th percentile
Date Fri, 17 Jun 2011 15:40:47 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051143#comment-13051143
] 

Shawn Heisey commented on SOLR-1972:
------------------------------------

I noticed what might be a problem.  Config:

  <lst name="rollingStatistics">
    <int name="history">604800</int>
    <int name="samples">16384</int>
    <arr name="percentiles">
      <int>90</int>
      <int>95</int>
      <int>99</int>
    </arr>
  </lst>

Current statistics generated by Solr 3.2 with this patch:

requests : 31861
errors : 0
timeouts : 0
totalTime : 454186
avgTimePerRequest : 14.255 
rollingRequests : 16384
rollingTotalTime : 106955
rollingAvgTimePerRequest : 6.528
rollingAvgRequestsPerSecond : -0.024
rollingMedian : 1
rolling90thPercentile : 9
rolling95thPercentile : 20
rolling99thPercentile : 96 

The requests per second is negative, which I think is a clear problem.

I am slightly concerned by seeing the median as 1 millisecond.  The shard has 9 million documents
in it and the index is 17GB, with only 9GB of RAM in the VM.  It might not actually be a problem
-- I was hitting it several times with a homegrown benchmarking app that has a relatively
limited (only a couple thousand) supply of search strings.  It's entirely possible that after
the first few thousand queries, everything was being served from Solr caches, and that what's
still in the rolling data store really did happen that quickly.


> Need additional query stats in admin interface - median, 95th and 99th percentile
> ---------------------------------------------------------------------------------
>
>                 Key: SOLR-1972
>                 URL: https://issues.apache.org/jira/browse/SOLR-1972
>             Project: Solr
>          Issue Type: Improvement
>    Affects Versions: 1.4
>            Reporter: Shawn Heisey
>            Priority: Minor
>         Attachments: SOLR-1972.patch, SOLR-1972.patch, SOLR-1972.patch
>
>
> I would like to see more detailed query statistics from the admin GUI.  This is what
you can get now:
> requests : 809
> errors : 0
> timeouts : 0
> totalTime : 70053
> avgTimePerRequest : 86.59209
> avgRequestsPerSecond : 0.8148785 
> I'd like to see more data on the time per request - median, 95th percentile, 99th percentile,
and any other statistical function that makes sense to include.  In my environment, the first
bunch of queries after startup tend to take several seconds each.  I find that the average
value tends to be useless until it has several thousand queries under its belt and the caches
are thoroughly warmed.  The statistical functions I have mentioned would quickly eliminate
the influence of those initial slow queries.
> The system will have to store individual data about each query.  I don't know if this
is something Solr does already.  It would be nice to have a configurable count of how many
of the most recent data points are kept, to control the amount of memory the feature uses.
 The default value could be something like 1024 or 4096.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message