hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6275) Race condition in FileOutputCommitter v2 for user-specified task output subdirs
Date Wed, 18 Mar 2015 13:36:39 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367128#comment-14367128
] 

Jason Lowe commented on MAPREDUCE-6275:
---------------------------------------

bq.I did not change the semantics of the file replacement even if the existing path a directory.
I am also not aware what is the background here, and tend to agree that we should have an
extra check. Probably in a separate JIRA.

Ah, yes, I missed this is basically existing behavior.  Agree we can tackle this in a separate
JIRA.

 bq. I don't think /tmp is hard coded here

It is hardcoded as the fallback for a missing test.build.data setting.  More specifically,
I think it should be something like this:

{code}
private static final Path outDir = new Path(
      System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")),
      TestFileOutputCommitter.class.getName());
{code}


> Race condition in FileOutputCommitter v2 for user-specified task output subdirs
> -------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6275
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6275
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Siqi Li
>            Assignee: Gera Shegalov
>            Priority: Critical
>         Attachments: MAPREDUCE-6275.002.patch, MAPREDUCE-6275.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message