lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrzej Bialecki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-4735) Improve Solr metrics reporting
Date Thu, 01 Dec 2016 13:49:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712011#comment-15712011
] 

Andrzej Bialecki  commented on SOLR-4735:
-----------------------------------------

Thanks Kelvin for creating the new tests! You beat me to it... :)

After some discussion with Shalin it looks like we need to be able to manage {{SolrMetricReporter}}-s
outside the {{SolrCore}} context - after all they need to also report metrics that are not
related to {{SolrCore}}:
* I added a notion of "group" of metrics, which corresponds to a top-level subsystem in a
Solr node (I'm not attached to the name, naming is hard, so if you have a better suggestion
please speak up):
{code}
  enum Group { jvm, jetty, http, node, core }
{code}
These correspond to JVM, Jetty, HttpClient, SolrCore, and node-level metrics such as Zookeeper
stats, Overseer queues etc. The idea is that if you fire JConsole you should see something
like this:
{code}
* solr
  - jvm
  - jetty
  - http
  - node
    - zookeeper
    - overseer
    ...
  - core
    - collection1
      - CORE
      - QUERYHANDLER
      …
{code}
* Since we need reporters for these top-level groups we need to be able to configure them
outside {{SolrCore}} config. On the other hand we probably don't want to always specify the
same reporters in each collection config, too. I'll try to refactor the code so that we can
specify the base reporter configs in {{CoreContainer}} configuration, and then override them
per collection if needed.
* Current implementation instantiates separate reporters for each collection. I'll look into
reusing single global-level reporters when possible, and creating new instances only if there
are per-collection overrides.
* Currently we use separate instances of {{MetricRegistry}} per core. This may seem wasteful
but if you consider deployments with thousands of cores, and multiply this by hundreds of
metrics per core then any lookups in a HashMap with 100k keys, or filtering by prefix, will
cause performance issues. So I think the current design is more scalable than using a single
registry for all cores.

> Improve Solr metrics reporting
> ------------------------------
>
>                 Key: SOLR-4735
>                 URL: https://issues.apache.org/jira/browse/SOLR-4735
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Alan Woodward
>            Assignee: Andrzej Bialecki 
>            Priority: Minor
>         Attachments: SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch
>
>
> Following on from a discussion on the mailing list:
> http://search-lucene.com/m/IO0EI1qdyJF1/codahale&subj=Solr+metrics+in+Codahale+metrics+and+Graphite+
> It would be good to make Solr play more nicely with existing devops monitoring systems,
such as Graphite or Ganglia.  Stats monitoring at the moment is poll-only, either via JMX
or through the admin stats page.  I'd like to refactor things a bit to make this more pluggable.
> This patch is a start.  It adds a new interface, InstrumentedBean, which extends SolrInfoMBean
to return a [[Metrics|http://metrics.codahale.com/manual/core/]] MetricRegistry, and a couple
of MetricReporters (which basically just duplicate the JMX and admin page reporting that's
there at the moment, but which should be more extensible).  The patch includes a change to
RequestHandlerBase showing how this could work.  The idea would be to eventually replace the
getStatistics() call on SolrInfoMBean with this instead.
> The next step would be to allow more MetricReporters to be defined in solrconfig.xml.
 The Metrics library comes with ganglia and graphite reporting modules, and we can add contrib
plugins for both of those.
> There's some more general cleanup that could be done around SolrInfoMBean (we've got
two plugin handlers at /mbeans and /plugins that basically do the same thing, and the beans
themselves have some weirdly inconsistent data on them - getVersion() returns different things
for different impls, and getSource() seems pretty useless), but maybe that's for another issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message