hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4128) AM Recovery expects all attempts of a completed task to also be completed.
Date Thu, 12 Apr 2012 14:49:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252468#comment-13252468
] 

Robert Joseph Evans commented on MAPREDUCE-4128:
------------------------------------------------

I ran the tests manually, but I ran them against all of Hadoop, not just mapreduce-project
so the counts for the -1s are likely to be off.

{noformat}
-1 overall.  Here are the results of testing the latest attachment
  http://issues.apache.org/jira/secure/attachment/12522297/MAPREDUCE-4128-1.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 2 new or modified test files.

    -1 javadoc.  The javadoc tool appears to have generated 14 warning messages.

    -1 javac.  The applied patch generated 1731 javac compiler warnings (more than the trunk's
current 1728 warnings).

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    -1 findbugs.  The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    -1 core tests.  The patch failed these unit tests:
                  org.apache.hadoop.tools.mapred.TestUniformSizeInputFormat
                  org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
                  org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree
                  org.apache.hadoop.mapreduce.v2.TestMRJobs
                  org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryEntities
                  org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
                  org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes

    +1 contrib tests.  The patch passed contrib unit tests.
{noformat}

The javadoc warnings look like they can be ignored.
The javac warnings need to be addressed though. Here is a diff between trunk, and the new
patch (yes they are copy and paste but we need to put in the @SurpressWarnings or something.
{noformat}
1279,1282c1279,1285
< [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[66,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
< [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[107,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
< [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[115,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
< [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[148,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
---
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[69,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[110,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[118,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[179,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[232,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[241,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
> [WARNING] /home/evans/src/commit/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestFetchFailure.java:[252,45]
[unchecked] unchecked call to handle(T) as a member of the raw type org.apache.hadoop.yarn.event.EventHandler
{noformat}

The findbugs warnings also look OK.  Nothing new, just the root config is off.

You may want to check the mr/yarn unit tests to be sure they are known issues.
                
> AM Recovery expects all attempts of a completed task to also be completed.
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4128
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4128
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 3.0.0
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>             Fix For: 3.0.0
>
>         Attachments: MAPREDUCE-4128-1.patch, MAPREDUCE-4128.patch
>
>
> The AM seems to assume that all attempts of a completed task (from a previous AM incarnation)
would also be completed. There is at least one case in which this does not hold. Case being
cancellation of a completed task resulting in a new running attempt.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message