hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9043) winutils can create unusable symlinks
Date Fri, 12 Apr 2013 19:14:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630513#comment-13630513

Chris Nauroth commented on HADOOP-9043:

I was just trying to resume progress on this to fix the remaining test failures related to
Windows symlinks on the local file system.  I discovered a new problem.  The trunk code still
has a Unix command dependency on the readlink command for determining the target of a symlink.
 (See {{RawLocalFs#readLink}}.)  This is the source of some of the test failures.

I propose the addition of a new "winutils readlink" command.  The logic of this command would

# Call {{CreateFile}} with {{OPEN_EXISTING}} and {{FILE_FLAG_OPEN_REPARSE_POINT}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx
# Call {{GetFileInformationByHandle}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa364952(v=vs.85).aspx
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa363788(v=vs.85).aspx
** http://msdn.microsoft.com/en-us/library/windows/desktop/gg258117(v=vs.85).aspx
# If not a reparse point, exit with code 1 and print nothing to stdout.  (This is what Unix
readlink does.)
# Call {{DeviceIoControl}} with {{FSCTL_GET_REPARSE_POINT}}.
** http://msdn.microsoft.com/en-us/library/aa363216(v=VS.85).aspx
** http://msdn.microsoft.com/en-us/library/aa364571.aspx
# Get the {{REPARSE_DATA_BUFFER}} structure.
** http://msdn.microsoft.com/en-us/library/ff552012.aspx
# Check if {{ReparseTag}} is {{IO_REPARSE_TAG_SYMLINK}}.
** http://msdn.microsoft.com/en-us/library/windows/desktop/aa365511(v=vs.85).aspx
# If not {{IO_REPARSE_TAG_SYMLINK}}, then...?
# Get target from {{SymbolicLinkReparseBuffer}}.
# Print target to stdout and exit with code 0.  (This is what Unix readlink does.)

Could someone with more Windows expertise review this and comment on whether or not the logic
looks correct?  There are a few edge cases that I'm not sure how to handle.  What should we
do if the reparse point is not a symlink (i.e. junction point)?  MSDN also mentions that some
reparse points may have a different data structure associated with them, a {{REPARSE_GUID_DATA_BUFFER}},
and I'm not sure what special handling is required around that.

> winutils can create unusable symlinks
> -------------------------------------
>                 Key: HADOOP-9043
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9043
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 3.0.0, 1-win
>            Reporter: Chris Nauroth
>            Assignee: Arpit Agarwal
>             Fix For: 3.0.0, 1-win
>         Attachments: HADOOP-9043.branch-1.2.patch, HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch,
> In general, the winutils symlink command rejects attempts to create symlinks targeting
a destination file that does not exist.  However, if given a symlink destination with forward
slashes pointing at a file that does exist, then it creates the symlink with the forward slashes,
and then attempts to open the file through the symlink will fail.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message