hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5470) LocalJobRunner does not work on Windows.
Date Thu, 22 Aug 2013 01:22:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13747129#comment-13747129
] 

Sandy Ryza commented on MAPREDUCE-5470:
---------------------------------------

Oh I see where I went wrong. I thought the task dirs were always placed under localJobDir
(which is used for writing the job conf) and that I was reusing code by using localJobDir
to construct the task path.  But they're actually actually in different places.  The localJobDir
is placed relative to current working directory, while the task dirs are placed under mapreduce.cluster.local.dir.

So either should move the task dirs under localJobDir or change back the part related to task
dirs.  It's not obvious to me which is better.
                
> LocalJobRunner does not work on Windows.
> ----------------------------------------
>
>                 Key: MAPREDUCE-5470
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5470
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Chris Nauroth
>
> {{LocalJobRunner#getLocalTaskDir}} creates a directory that is unique to the task ID.
 The logic of this method concatenates the local job dir and a task-specific path, but one
of the arguments is a {{Path}} with a scheme, so the final result has "file:" embedded in
it.  This works on Linux, but the ':' is an invalid character in a file name on Windows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message