hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8409) Address Hadoop path related issues on Windows
Date Wed, 23 May 2012 20:21:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281875#comment-13281875
] 

Daryn Sharp commented on HADOOP-8409:
-------------------------------------

bq. Aksing users to enter input paths in form "c:/some/path" does not seem like the right
thing to do.  Please let me know if you agree with me here. 
Unfortunately I disagree... 

HADOOP-8139 has a long discussion, but the short summary is:  Hadoop uses URIs, so the hadoop
veterans decided the hacky support for \ needs to be removed.  Although "dir/file" might look
like a native path, it's still a URI being prepending with the default fs URI.  I spun off
HADOOP-8164 to fix the meta-char quoting issue caused by the windows hack -- but bug persists
on windows!

HADOOP-8139 was linked to the window umbrella jira (HADOOP-8079) over a year ago, with the
reporter Alexander quoted as "We are only using backslashes () for escaping meta-characters
and not as alternative path-separators. So, things should work in a uniform manner."

Of note, it looks like for at least the past 6 years, MS has said URIs using \\ is deprecated?
 http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx

bq.  Actually, this does not work for paths that are symlinks.  For example, new Path("/some/path#symlink")
will encode the "#" character internally, so we lose the symlink behavior.

I'm sorry, but you've completely lost me on how URI fragments are related to symlinks?  Is
this a windows convention?
                
> Address Hadoop path related issues on Windows
> ---------------------------------------------
>
>                 Key: HADOOP-8409
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8409
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, test, util
>    Affects Versions: 1.0.0
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-8409-branch-1-win.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> There are multiple places in prod and test code where Windows paths are not handled properly.
From a high level this could be summarized with:
> 1. Windows paths are not necessarily valid DFS paths (while Unix paths are)
> 2. Windows paths are not necessarily valid URIs (while Unix paths are)
> #1 causes a number of tests to fail because they implicitly assume that local paths are
valid DFS paths (by extracting the DFS test path from for example "test.build.data" property)
> #2 causes issues when URIs are directly created on path strings passed in by the user

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message