hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4987) TestMRJobs#testDistributedCache fails on Windows due to unexpected behavior of symlinks
Date Fri, 08 Feb 2013 00:21:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574097#comment-13574097
] 

Ivan Mitic commented on MAPREDUCE-4987:
---------------------------------------

Thanks for reporting this Chris.

bq. But symlinks CAN be used for functional purposes i.e linking to libraries etc. ?
Hi Vinod. I believe we'll have to port the branch-1-win semantic to trunk to properly support
symlinks on both Java6 and Java7 on Windows. Yes, symlinks can be created to point to folders
and files, however, Java6 does not interpret them correctly. We've seen so many issues with
symlinks on Java6, and the only option that worked fine (and was signed off on) is to do a
file copy in case of Java6. HADOOP-9061 talks about some of these problems.

bq. If so we can just do the platform check in the test-case.
We also initially thought this would be fine (you can check thru branch-1-win history :)).
However, the real problem comes when someone tries to access the symlink thru Java APIs. Examples
of problems are, File#length on symlinks returns zero. This means that RLFS does not work
on top of symlinks. Additionally, File#renameTo on symlink renames the target file instead
of the symlink (really strange I know :)). 

Hope this helps
                
> TestMRJobs#testDistributedCache fails on Windows due to unexpected behavior of symlinks
> ---------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4987
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4987
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distributed-cache, nodemanager
>    Affects Versions: trunk-win
>            Reporter: Chris Nauroth
>
> On Windows, {{TestMRJobs#testDistributedCache}} fails on an assertion while checking
the length of a symlink.  It expects to see the length of the target of the symlink, but Java
6 on Windows always reports that a symlink has length 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message