hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-220) Collecting cpu and memory usage for MapReduce tasks
Date Tue, 03 Aug 2010 18:54:24 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894990#action_12894990
] 

Scott Chen commented on MAPREDUCE-220:
--------------------------------------

Hey Philip,

We haven't try test this under the case of JVM re-use. But I think you are right about this.
We need to do some more work for this case.

We can still get the correct PID in JVM reuse case. Because we use
{code}
String pid = System.getenv().get("JVM_PID");
{code}
which is invoked from Task.updateCounters().
So we should be able to get the correct PID for the task no matter JVM is reused or not.

The problem is the cumulated CPU time. Because the process may be used by another task for
a while.
One way to solve this is to send only the current value instead of cumulated value.
Does this sound correct to you?

Scott

> Collecting cpu and memory usage for MapReduce tasks
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-220
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-220
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: task, tasktracker
>            Reporter: Hong Tang
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-220-20100616.txt, MAPREDUCE-220-v1.txt, MAPREDUCE-220.txt
>
>
> It would be nice for TaskTracker to collect cpu and memory usage for individual Map or
Reduce tasks over time.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message