hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8409) Address Hadoop path related issues on Windows
Date Mon, 21 May 2012 22:23:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280555#comment-13280555

Ivan Mitic commented on HADOOP-8409:

Thanks for the feedback, again!

The problem is that Windows paths have the following "natural" form: "c:\some\path" in which
case the URI creation fails. I actually believe that this is a good change as it simplifies
the code on the call site (why having to create an URI first before creating a Path?). On
the other hand, if you can see some problems that could come out of this, that would be of
great help. Let me know if this makes sense. 

getUriPath: The problematic URI would be the following: "file://test:8000/c:/some/path" in
which case the path value would be "/c:/some/path" what does not work well for shell. I will
likely revisit this specific problem, as I believe that we can find a better solution.

/tmp: would it make sense to have the test name embedded in the DFS path? That should solve
the concurrent problem.
> Address Hadoop path related issues on Windows
> ---------------------------------------------
>                 Key: HADOOP-8409
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8409
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, test, util
>    Affects Versions: 1.0.0
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-8409-branch-1-win.patch
>   Original Estimate: 168h
>  Remaining Estimate: 168h
> There are multiple places in prod and test code where Windows paths are not handled properly.
From a high level this could be summarized with:
> 1. Windows paths are not necessarily valid DFS paths (while Unix paths are)
> 2. Windows paths are not necessarily valid URIs (while Unix paths are)
> #1 causes a number of tests to fail because they implicitly assume that local paths are
valid DFS paths (by extracting the DFS test path from for example "test.build.data" property)
> #2 causes issues when URIs are directly created on path strings passed in by the user

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message