Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Thu, 1 Dec 2016 13:49:58 +0000 (UTC)
From: "Andrzej Bialecki  (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <JIRA.12643182.1366284052000.412652.1480600198657@Atlassian.JIRA>
In-Reply-To: <JIRA.12643182.1366284052000@Atlassian.JIRA>
References: <JIRA.12643182.1366284052000@Atlassian.JIRA> <JIRA.12643182.1366284052833@arcas>
Subject: [jira] [Commented] (SOLR-4735) Improve Solr metrics reporting
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 01 Dec 2016 13:50:00 -0000


    [ https://issues.apache.org/jira/browse/SOLR-4735?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15712=
011#comment-15712011 ]=20

Andrzej Bialecki  commented on SOLR-4735:
-----------------------------------------

Thanks Kelvin for creating the new tests! You beat me to it... :)

After some discussion with Shalin it looks like we need to be able to manag=
e {{SolrMetricReporter}}-s outside the {{SolrCore}} context - after all the=
y need to also report metrics that are not related to {{SolrCore}}:
* I added a notion of "group" of metrics, which corresponds to a top-level =
subsystem in a Solr node (I'm not attached to the name, naming is hard, so =
if you have a better suggestion please speak up):
{code}
  enum Group { jvm, jetty, http, node, core }
{code}
These correspond to JVM, Jetty, HttpClient, SolrCore, and node-level metric=
s such as Zookeeper stats, Overseer queues etc. The idea is that if you fir=
e JConsole you should see something like this:
{code}
* solr
  - jvm
  - jetty
  - http
  - node
    - zookeeper
    - overseer
    ...
  - core
    - collection1
      - CORE
      - QUERYHANDLER
      =E2=80=A6
{code}
* Since we need reporters for these top-level groups we need to be able to =
configure them outside {{SolrCore}} config. On the other hand we probably d=
on't want to always specify the same reporters in each collection config, t=
oo. I'll try to refactor the code so that we can specify the base reporter =
configs in {{CoreContainer}} configuration, and then override them per coll=
ection if needed.
* Current implementation instantiates separate reporters for each collectio=
n. I'll look into reusing single global-level reporters when possible, and =
creating new instances only if there are per-collection overrides.
* Currently we use separate instances of {{MetricRegistry}} per core. This =
may seem wasteful but if you consider deployments with thousands of cores, =
and multiply this by hundreds of metrics per core then any lookups in a Has=
hMap with 100k keys, or filtering by prefix, will cause performance issues.=
 So I think the current design is more scalable than using a single registr=
y for all cores.

> Improve Solr metrics reporting
> ------------------------------
>
>                 Key: SOLR-4735
>                 URL: https://issues.apache.org/jira/browse/SOLR-4735
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Alan Woodward
>            Assignee: Andrzej Bialecki=20
>            Priority: Minor
>         Attachments: SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, S=
OLR-4735.patch
>
>
> Following on from a discussion on the mailing list:
> http://search-lucene.com/m/IO0EI1qdyJF1/codahale&subj=3DSolr+metrics+in+C=
odahale+metrics+and+Graphite+
> It would be good to make Solr play more nicely with existing devops monit=
oring systems, such as Graphite or Ganglia.  Stats monitoring at the moment=
 is poll-only, either via JMX or through the admin stats page.  I'd like to=
 refactor things a bit to make this more pluggable.
> This patch is a start.  It adds a new interface, InstrumentedBean, which =
extends SolrInfoMBean to return a [[Metrics|http://metrics.codahale.com/man=
ual/core/]] MetricRegistry, and a couple of MetricReporters (which basicall=
y just duplicate the JMX and admin page reporting that's there at the momen=
t, but which should be more extensible).  The patch includes a change to Re=
questHandlerBase showing how this could work.  The idea would be to eventua=
lly replace the getStatistics() call on SolrInfoMBean with this instead.
> The next step would be to allow more MetricReporters to be defined in sol=
rconfig.xml.  The Metrics library comes with ganglia and graphite reporting=
 modules, and we can add contrib plugins for both of those.
> There's some more general cleanup that could be done around SolrInfoMBean=
 (we've got two plugin handlers at /mbeans and /plugins that basically do t=
he same thing, and the beans themselves have some weirdly inconsistent data=
 on them - getVersion() returns different things for different impls, and g=
etSource() seems pretty useless), but maybe that's for another issue.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org