hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4099) ApplicationMaster may fail to remove staging directory
Date Wed, 11 Apr 2012 14:45:19 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251639#comment-13251639
] 

Hadoop QA commented on MAPREDUCE-4099:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522250/MAPREDUCE-4099-addendum.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified test files.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    -1 core tests.  The patch failed these unit tests:
                  org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2198//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2198//console

This message is automatically generated.
                
> ApplicationMaster may fail to remove staging directory
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-4099
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4099
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.2
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Critical
>             Fix For: 0.23.3, 2.0.0
>
>         Attachments: MAPREDUCE-4099-addendum.patch, MAPREDUCE-4099-addendum.patch, MAPREDUCE-4099.patch,
MAPREDUCE-4099.patch, MAPREDUCE-4099.patch
>
>
> When the ApplicationMaster shuts down it's supposed to remove the staging directory,
assuming properties weren't set to override this behavior. During shutdown the AM tells the
ResourceManager that it has finished before it cleans up the staging directory.  However upon
hearing the AM has finished, the RM turns right around and kills the AM container.  If the
AM is too slow, the AM will be killed before the staging directory is removed.
> We're seeing the AM lose this race fairly consistently on our clusters, and the lack
of staging directory cleanup quickly leads to filesystem quota issues for some users.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message