hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8409) Address Hadoop path related issues on Windows
Date Wed, 06 Jun 2012 22:21:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290503#comment-13290503

Ivan Mitic commented on HADOOP-8409:

Thanks Daryn, that makes sense. We finally decided not to further extend the support for backslash.
This means that input paths to GenericOptionsParser (files/archives/libjars) must be valid
URIs on Windows! 

I will repurpose this Jira to fix test failures in TestGenericOptionsParser and TestCommandLineJobSubmission
in line with this decision.

Also, I have created two new Jiras based on the original patch attached here, as these issue
can generally stand on their own:
 - [HADOOP-8487|https://issues.apache.org/jira/browse/HADOOP-8409]
 - [MAPREDUCE-4321|https://issues.apache.org/jira/browse/MAPREDUCE-4321]

Separately, we are still discussing possible solutions for [HADOOP-8139|https://issues.apache.org/jira/browse/HADOOP-8139].
My goal was to break down the original problem into smaller isolated pieces so that we can
make incremental progress in parallel.
> Address Hadoop path related issues on Windows
> ---------------------------------------------
>                 Key: HADOOP-8409
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8409
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, test, util
>    Affects Versions: 1.0.0
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-8409-branch-1-win.patch
>   Original Estimate: 168h
>  Remaining Estimate: 168h
> There are multiple places in prod and test code where Windows paths are not handled properly.
From a high level this could be summarized with:
> 1. Windows paths are not necessarily valid DFS paths (while Unix paths are)
> 2. Windows paths are not necessarily valid URIs (while Unix paths are)
> #1 causes a number of tests to fail because they implicitly assume that local paths are
valid DFS paths (by extracting the DFS test path from for example "test.build.data" property)
> #2 causes issues when URIs are directly created on path strings passed in by the user

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message