hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (MAPREDUCE-6512) FileOutputCommitter tasks unconditionally create parent directories
Date Mon, 11 Apr 2016 19:05:25 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Badger reassigned MAPREDUCE-6512:
--------------------------------------

    Assignee: Chang Li  (was: Eric Badger)

> FileOutputCommitter tasks unconditionally create parent directories
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6512
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Chang Li
>            Assignee: Chang Li
>         Attachments: MAPREDUCE-6512.2.patch, MAPREDUCE-6512.2.patch, MAPREDUCE-6512.patch,
MAPREDUCE-6512.patch
>
>
> If the output directory is deleted then subsequent tasks should fail. Instead they blindly
create the missing parent directories, leading the job to be "succesful" despite potentially
missing almost all of the output. Task attempts should fail if the parent app attempt directory
is missing when they go to create their task attempt directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message