hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1143) runningMapTasks counter is not properly decremented in case of failed Tasks.
Date Mon, 09 Nov 2009 11:22:32 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774942#action_12774942

Hemanth Yamijala commented on MAPREDUCE-1143:

Not really about this patch, but the logic for task status updates is way too complicated
to reason correctly. For instance, even after this patch, I see that the number of running
tasks is decremented under different checks when a task completes and when a task fails. I
assume this is for good reason, but still it is difficult to review. 

A radically different idea, which might be completely wrong (*smile*):

Can we assume this invariant: 
- A task attempt is added to the list of active tasks in TaskInProgress whenever it is scheduled.
Running task counts should be incremented in this code path.
- Likewise, a decrement of running tasks should happen whenever the task attempt being reported
as completed or failed is removed from the active tasks list.

At least to me, this appears simpler to understand, and fixing the issue reported. Does this
make sense ? Anything wrong in the logic ?

> runningMapTasks counter is not properly decremented in case of failed Tasks.
> ----------------------------------------------------------------------------
>                 Key: MAPREDUCE-1143
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1143
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: rahul k singh
>            Priority: Blocker
>         Attachments: MAPRED-1143-1.patch, MAPRED-1143-2.patch, MAPRED-1143-2.patch, MAPRED-1143-3.patch,
MAPRED-1143-4.patch, MAPRED-1143-ydist-1.patch, MAPRED-1143-ydist-2.patch, MAPRED-1143-ydist-3.patch,
MAPRED-1143-ydist-4.patch, MAPRED-1143-ydist-5.patch, MAPRED-1143-ydist-6.patch

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message