hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3982) TestEmptyJob fails with FileNotFound
Date Thu, 08 Mar 2012 16:45:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225294#comment-13225294
] 

Robert Joseph Evans commented on MAPREDUCE-3982:
------------------------------------------------

That would not really work.  The directory structure is the following.

<OUTPUT>/_temporary/<ATTEMPT_ID>/...

The problem was that the code assumed that <OUTPUT>/_temporary/<ATTEMPT_ID>/ was
being created by the individual map/reduce tasks.  If it was not then other code would blowup,
so instead of just creating <OUTPUT>/_temporary in the setup from before we are now
creating <OUTPUT>/_temporary/<ATTEMPT_ID>/

When we clean up from a job we still want to completely remove the _temporary directory, not
just <OUTPUT>/_temporary/<ATTEMPT_ID>/ or there will be an extra directory sitting
around in the <OUTPUT> directory that will cause other problems.
                
> TestEmptyJob fails with FileNotFound
> ------------------------------------
>
>                 Key: MAPREDUCE-3982
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3982
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.24.0, 0.23.2
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Critical
>         Attachments: MR-3982.txt
>
>
> TestEmptyJob fails because teh FileOutputCommitter expects a directory to be created
that is not created.  The FileOutputCommitter should either ignore the error or create the
directory itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message