hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7507) jvm metrics all use the same namespace
Date Thu, 01 Sep 2011 18:50:10 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095524#comment-13095524
] 

Todd Lipcon commented on HADOOP-7507:
-------------------------------------

Thinking about this a bit... it seems like doing the generic tag handling won't work well
in Ganglia, given that the JT has job-specific metrics tagged by the job ID (eg fairscheduler
job level metrics). Over time, the JT will accumulate an unbounded number of metrics in ganglia,
and I don't think Ganglia ever "ages off" a gmetric that it hasn't seen in a while. I think
we need some sort of config for the ganglia context that says which tag names need to be included
as part of a metric (and perhaps which metrics shouldn't be reported at all in ganglia?)

> jvm metrics all use the same namespace
> --------------------------------------
>
>                 Key: HADOOP-7507
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7507
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: metrics
>    Affects Versions: 0.20.2
>            Reporter: Jeff Bean
>            Assignee: Alejandro Abdelnur
>             Fix For: 0.23.0
>
>         Attachments: HADOOP-7507-v2.patch, HADOOP-7507v1.patch, HADOOP-7507v3.patch,
HADOOP-7507v4.patch, HADOOP-7507v5.patch, JvmMetrics.java, hadoop-metrics.properties, screenshot-1.jpg
>
>
> Ganglia jvm metrics don't make sense because it's not clear which java process the metrics
refer to. In fact, all hadoop java processes running on a node report their jvm metrics to
the same namespace.
> The metrics are exposed by the "jvm" context in JvmMetrics.java. This leads to confusing
and nonsensical graphs in ganglia and maybe other monitoring tools.
> One way to fix this is to make sure the process name is reported in the jvm context,
making it clear which process is associated with the context, and separating out the jvm metrics
per process.
> This is marked as an "incompatible change" because the fix provided removes the JVM metrics
and replaces it with process-specific metrics.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message