hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4607) Race condition in ReduceTask completion can result in Task being incorrectly failed
Date Sat, 01 Sep 2012 00:57:07 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446548#comment-13446548

Hadoop QA commented on MAPREDUCE-4607:

+1 overall.  Here are the results of testing the latest attachment 
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 2 new or modified test files.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit

    +1 core tests.  The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2807//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2807//console

This message is automatically generated.
> Race condition in ReduceTask completion can result in Task being incorrectly failed
> -----------------------------------------------------------------------------------
>                 Key: MAPREDUCE-4607
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4607
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.1.0-alpha
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: MAPREDUCE-4607.1.patch, MAPREDUCE-4607.2.patch
> Problem reported by chackaravarthy in MAPREDUCE-4252
> This problem has been handled when speculative task launched for map task and other attempt
got failed (not killed)
> Can the similar kind of scenario can happen in case of reduce task?
> Consider the following scenario for reduce task in case of speculation (one attempt got
> 1. A task attempt is started.
> 2. A speculative task attempt for the same task is started.
> 3. The first task attempt completes and causes the task to transition to SUCCEEDED.
> 4. Then speculative task attempt will be killed because of the completion of first attempt.
> As a result, internal error will be thrown from this attempt (TaskImpl.MapRetroactiveKilledTransition)
and hence task attempt failure leads to job failure.
> TaskImpl.MapRetroactiveKilledTransition
> if (!TaskType.MAP.equals(task.getType())) {
>         LOG.error("Unexpected event for REDUCE task " + event.getType());
>         task.internalError(event.getType());
>       }
> So, do we need to have following code in MapRetroactiveKilledTransition also just like
in MapRetroactiveFailureTransition.
> if (event instanceof TaskTAttemptEvent) {
>         TaskTAttemptEvent castEvent = (TaskTAttemptEvent) event;
>         if (task.getState() == TaskState.SUCCEEDED &&
>             !castEvent.getTaskAttemptID().equals(task.successfulAttempt)) {
>           // don't allow a different task attempt to override a previous
>           // succeeded state
>           return TaskState.SUCCEEDED;
>         }
>       }
> please check whether this is a valid case and give your suggestion.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message