hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-901) Move Framework Counters into a TaskMetric structure
Date Tue, 25 Aug 2009 20:29:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747627#action_12747627
] 

Owen O'Malley commented on MAPREDUCE-901:
-----------------------------------------

Comments:
   1. I'd replace the array of longs with explicit fields.
   2. I'd drop the booleans for whether it has been used.
   3. The serialization should just be to write the fields in order using vints.
   4. updateFrameworkCounters should just update the counters by using the enum values explicitly.
   5. updateFrameworkCounters should set, not increment the related counter.
   6. TaskMetrics should be a field in the TaskStatus.

> Move Framework Counters into a TaskMetric structure
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-901
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-901
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task
>    Affects Versions: 0.21.0
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>             Fix For: 0.21.0
>
>         Attachments: 901_1.patch, 901_1.patch
>
>
> I think we should move all of the Counters that the framework updates into a single class
called TaskMetrics. TaskMetrics would have specific fields for each of the metrics like input
records, input bytes, output records, etc.
> It would both reduce the serialized size of the heartbeats (by shrinking the Counters
down to just the user's counters) and decrease the latency for updates to the JobTracker (since
Counters are sent at most 1/minute instead of 1/heartbeat).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message