hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8409) Address Hadoop path related issues on Windows
Date Mon, 21 May 2012 23:06:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280592#comment-13280592
] 

Daryn Sharp commented on HADOOP-8409:
-------------------------------------

bq.  The problem is that Windows paths have the following "natural" form: "c:\some\path" in
which case the URI creation fails.
After a long discussion in HADOOP-8139, it was decided that only RFC standard URIs will be
supported by hadoop.  Paths using "\" are not going to be supported.

bq. why having to create an URI first before creating a Path?
There's no reason you have to, you can always use {{new Path(String)}}.  Although {{new Path(File#toUri())}}
might be useful.  I don't have access to windows, but I thought {{File}} considered {{/c:/some/path}}
to be a valid path?

bq. /tmp: would it make sense to have the test name embedded in the DFS path? That should
solve the concurrent problem.

{{System.getProperty("test.build.dir", "build/test")}} is used pretty pervasively by the tests.
 Is there a complication with using it for these tests?

                
> Address Hadoop path related issues on Windows
> ---------------------------------------------
>
>                 Key: HADOOP-8409
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8409
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, test, util
>    Affects Versions: 1.0.0
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-8409-branch-1-win.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> There are multiple places in prod and test code where Windows paths are not handled properly.
From a high level this could be summarized with:
> 1. Windows paths are not necessarily valid DFS paths (while Unix paths are)
> 2. Windows paths are not necessarily valid URIs (while Unix paths are)
> #1 causes a number of tests to fail because they implicitly assume that local paths are
valid DFS paths (by extracting the DFS test path from for example "test.build.data" property)
> #2 causes issues when URIs are directly created on path strings passed in by the user

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message