hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4987) TestMRJobs#testDistributedCache fails on Windows due to unexpected behavior of symlinks
Date Fri, 15 Mar 2013 19:16:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603718#comment-13603718

Arpit Agarwal commented on MAPREDUCE-4987:


Chris explained to me offline about the change in FileUtil#createJarWithClassPath. Quoting
here since I found it helpful to understand the change.

In the method sanitizeEnv, you'll see that nodemanager does various things to set up a new
environment for the container to be launched.  The final state of this environment will be
different from the environment of the currently running process (the nodemanager itself).

The most glaring problem with this bug was the setting of PWD to the new container work directory.
 There are various classpath entries for the distributed cache files that are of the form
$PWD/file on Mac or %PWD%/file on Windows, and FileUtil#createJarWithClassPath needs to expand
this to <container_dir>/file.  Without this change, the variable expansion would be
incorrect: <nodemanager_working_dir>/file on Mac or just /file on Windows (since Windows
doesn't intrinsically have %PWD% defined until nodemanager sets it in sanitizeEnv).
> TestMRJobs#testDistributedCache fails on Windows due to unexpected behavior of symlinks
> ---------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-4987
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4987
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distributed-cache, nodemanager
>    Affects Versions: 3.0.0
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: MAPREDUCE-4987.1.patch
> On Windows, {{TestMRJobs#testDistributedCache}} fails on an assertion while checking
the length of a symlink.  It expects to see the length of the target of the symlink, but Java
6 on Windows always reports that a symlink has length 0.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message