hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6728) Overhaul metrics framework
Date Sat, 18 Sep 2010 00:26:42 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910871#action_12910871
] 

Luke Lu commented on HADOOP-6728:
---------------------------------

@Eric Yang:

The framework is designed mostly to collect and dispatch per process metrics to monitor overall
status (aggregates) of the system. While it's certainly possible to send per job metrics through
this framework (via clever use of record and metrics/tagging naming conventions), the o.a.h.mapred.Reporter/Counters
stuff is not really the target use case, as the latter support nested counter groups more
naturally.

As for the API to query metrics from MetricsSource, any generic JMX client should work, as
the metrics are just exported as attributes of an MBean (which corresponds to a MetricsSource.)
E.g., jmx4perl package have a jmx to http agent that can act as an http gateway if that's
what you want.



> Overhaul metrics framework
> --------------------------
>
>                 Key: HADOOP-6728
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6728
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 0.20.2
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>             Fix For: 0.22.0
>
>         Attachments: hadoop-6728-y20.104.patch, metrics1-flow.png, metrics2-builder-r2.png,
metrics2-builder.png, metrics2-flow.png, metrics2-mutable-r2.png, metrics2-mutable.png, metrics2-record-r2.png,
metrics2-record.png, metrics2-uml-r2.png, metrics2-uml.png
>
>
> Per discussions with Arun, Chris, Hong and Rajiv, et al, we concluded that the current
metrics framework needs an overhaul to:
> * Allow multiple plugins for different monitoring systems simultaneously (see also: HADOOP-6508).
> * Refresh metrics plugin config without server restart.
> ** Including filtering of metrics per plugin.
> * Support metrics schema for plugins.
> The jira will be resolved when core hadoop components (hdfs, mapreduce) are updated to
use the new framework . Updates to external components that use the existing metrics framework
will be tracked by different issues.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message