hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3982) TestEmptyJob fails with FileNotFound
Date Thu, 08 Mar 2012 16:45:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225294#comment-13225294

Robert Joseph Evans commented on MAPREDUCE-3982:

That would not really work.  The directory structure is the following.


The problem was that the code assumed that <OUTPUT>/_temporary/<ATTEMPT_ID>/ was
being created by the individual map/reduce tasks.  If it was not then other code would blowup,
so instead of just creating <OUTPUT>/_temporary in the setup from before we are now
creating <OUTPUT>/_temporary/<ATTEMPT_ID>/

When we clean up from a job we still want to completely remove the _temporary directory, not
just <OUTPUT>/_temporary/<ATTEMPT_ID>/ or there will be an extra directory sitting
around in the <OUTPUT> directory that will cause other problems.
> TestEmptyJob fails with FileNotFound
> ------------------------------------
>                 Key: MAPREDUCE-3982
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3982
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0, 0.23.2
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Critical
>         Attachments: MR-3982.txt
> TestEmptyJob fails because teh FileOutputCommitter expects a directory to be created
that is not created.  The FileOutputCommitter should either ignore the error or create the
directory itself.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message