hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Franklin Ta (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6874) JvmMetrics can't distinguish between jvms with same processName
Date Thu, 22 Jul 2010 05:45:49 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Franklin Ta updated HADOOP-6874:
--------------------------------

    Description: 
JvmMetrics has three tags: hostName, processName, and sessionId. For processes such as tasktracker/jobtracker/namenode/datanode
which there is only one of on each host, these tags are fine. But for process names such as
"MAP" and "REDUCE", since there might be multiple jvms running map/reduce tasks, we might
end up with multiple set of metrics which all have the same tags, and no way to find out which
jvm they actually correspond to. (In addition, since there is jvm reuse, those process names
might not correspond to the actual task being ran)

A quick fix is to change this line in Child.java
JvmMetrics.init(task.getPhase().toString(), job.getSessionId());
to this
JvmMetrics.init(jvmId.toString(), job.getSessionId());
so that we are using the jvm id for the process name instead.

  was:
JvmMetrics has three tags: hostName, processName, and sessionId. For processes such tasktracker/jobtracker/namenode/datanode
which there is only one of on each host, these tags are fine. But for process names such as
"MAP" and "REDUCE", since there might be multiple jvms running map/reduce tasks, we might
end up with multiple set of metrics which all have the same tags, and no way to find out which
jvm they actually correspond to. (In addition, since there is jvm reuse, those process names
might not correspond to the actual task being ran)

A quick fix is to change this line in Child.java
JvmMetrics.init(task.getPhase().toString(), job.getSessionId());
to this
JvmMetrics.init(jvmId.toString(), job.getSessionId());
so that we are using the jvm id for the process name instead.


> JvmMetrics can't distinguish between jvms with same processName
> ---------------------------------------------------------------
>
>                 Key: HADOOP-6874
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6874
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: metrics
>    Affects Versions: 0.20.2
>            Reporter: Franklin Ta
>            Priority: Minor
>
> JvmMetrics has three tags: hostName, processName, and sessionId. For processes such as
tasktracker/jobtracker/namenode/datanode which there is only one of on each host, these tags
are fine. But for process names such as "MAP" and "REDUCE", since there might be multiple
jvms running map/reduce tasks, we might end up with multiple set of metrics which all have
the same tags, and no way to find out which jvm they actually correspond to. (In addition,
since there is jvm reuse, those process names might not correspond to the actual task being
ran)
> A quick fix is to change this line in Child.java
> JvmMetrics.init(task.getPhase().toString(), job.getSessionId());
> to this
> JvmMetrics.init(jvmId.toString(), job.getSessionId());
> so that we are using the jvm id for the process name instead.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message