hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4252) MR2 job never completes with 1 pending task
Date Wed, 29 Aug 2012 20:31:08 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444383#comment-13444383

Bikas Saha commented on MAPREDUCE-4252:

We could move the check for TaskType after the check for successful task attempt has been
made. This is similar to the pattern for MapRetroActiveFailureTransition(). So the fix is
a simple downward movement of the check for TaskType.

The check for whether the task is MAP should still be there (and is also there for MapRetroActiveFailureTransition())
because after a REDUCE has completed with SUCCEEDED there is no reason to fail it because
its outputs are already preserved in HDFS. So any attempt to fail it would be a bug in the

We could also potentially improve the check of a successful attempt in MapRetroActiveKilledTransition()
by using the condition in MapRetroActiveFailureTransition().

I opened YARN-61 to track this. Currently assigned to me because I wrote MapRetroactiveKilledTransition.
[~chaku88] Feel free to assign this to yourself.

> MR2 job never completes with 1 pending task
> -------------------------------------------
>                 Key: MAPREDUCE-4252
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4252
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1
>            Reporter: Tom White
>            Assignee: Tom White
>             Fix For: 0.23.3, 2.1.0-alpha, 3.0.0
>         Attachments: MAPREDUCE-4252.patch, MAPREDUCE-4252.patch, MAPREDUCE-4252.patch,
MAPREDUCE-4252.patch, MapReduce.png
> This was found by ATM:
> bq. I ran a teragen with 1000 map tasks. Many task attempts failed, but after 999 of
the tasks had completed, the job is now sitting forever with 1 task "pending".

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message