hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Bowen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1446) Metrics from the TaskTracker are updated only when map/reduce tasks start/end/fail
Date Fri, 01 Jun 2007 22:08:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500873

David Bowen commented on HADOOP-1446:

The TaskTracker fields mapTotal and reduceTotal now need to be declared volatile, since they
are being accessed from the Metrics thread.

Also, completeTask is wrong.  Actually, it was wrong before your changes (my bad).  Now, as
soon as one task has been completed, it will report one task being completed every blurb period
regardless of the actual number. The suggested fix is to add a numCompletedTasks field to
TaskTrackerMetrics, and make completeTask simply increment that field. Then doUpdates should
call incrMetric(numCompletedTasks), and afterwards set numCompletedTasks=0.

In general, all the metrics in a MetricsRecord should be updated in the same place in the
code, and update() should be called immediately afterwards.  

> Metrics from the TaskTracker are updated only when map/reduce tasks start/end/fail
> ----------------------------------------------------------------------------------
>                 Key: HADOOP-1446
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1446
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.12.3
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>             Fix For: 0.14.0
>         Attachments: 1446.patch
> Metrics reporting from the TaskTracker for the maps_running and reduces_running metrics
are done only when a task is starting up or finishing up. This is wrong and the metrics reporting
should be done periodically during the life of the TaskTracker just as it is done for the
JobTracker case.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message