hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6728) Overhaul metrics framework
Date Sat, 21 Aug 2010 02:07:20 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Luke Lu updated HADOOP-6728:
----------------------------

    Attachment: hadoop-6728-y20.104.patch

The patch is a combined (commons/mapreduce/hdfs) patch against yahoo-hadoop-0.20.104 (published
at http://github.com/yahoo/hadoop-common/). The code should be quite stable (based on ongoing
scale testing.)

The trunk patch will include significant API improvement with annotations (not included in
the above patch). The design notes have been updated to reflect the current code: [rev2|http://wiki.apache.org/hadoop/HADOOP-6728-MetricsV2?action=recall&rev=2]

Note, the patch completely removes the old metrics framework, which is appropriate for our
use cases (and forced porting of everything). The main question for the community is: do we
want to provide a mechanism to have the two framework coexists in 0.22, where the metrics
framework can be switched at runtime or let's move everything to v2 in 0.22?

The answer to the question would alter the trunk patch significantly.

I prefer the latter as the former (dual framework switchable at runtime per process) would
cause more confusion to users (and more throwaway work for me.). As an incentive for the latter,
I offer to port ganglia 3.1+ plugin for the new framework, even though we don't use it :)


> Overhaul metrics framework
> --------------------------
>
>                 Key: HADOOP-6728
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6728
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 0.20.2
>            Reporter: Luke Lu
>            Assignee: Luke Lu
>             Fix For: 0.22.0
>
>         Attachments: hadoop-6728-y20.104.patch, metrics1-flow.png, metrics2-builder-r2.png,
metrics2-builder.png, metrics2-flow.png, metrics2-mutable-r2.png, metrics2-mutable.png, metrics2-record-r2.png,
metrics2-record.png, metrics2-uml-r2.png, metrics2-uml.png
>
>
> Per discussions with Arun, Chris, Hong and Rajiv, et al, we concluded that the current
metrics framework needs an overhaul to:
> * Allow multiple plugins for different monitoring systems simultaneously (see also: HADOOP-6508).
> * Refresh metrics plugin config without server restart.
> ** Including filtering of metrics per plugin.
> * Support metrics schema for plugins.
> The jira will be resolved when core hadoop components (hdfs, mapreduce) are updated to
use the new framework . Updates to external components that use the existing metrics framework
will be tracked by different issues.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message