hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4632) globStatus using backslash for escaping does not work on Windows
Date Thu, 15 Aug 2013 06:05:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740732#comment-13740732

Ivan Mitic commented on HDFS-4632:

I am also fine with Chris’ proposal #1 from above. Changing path behavior would indeed break
many downstream projects. The fact that java.io.File converts all paths to backslash-paths
on Windows additionally complicates things. 

I also did investigation on Windows paths in Hadoop a while back, and this issue came up.
On top of the above 3 proposals, we have 2 other options:
4. Replace backslash with a forward slash only if the backslash is not followed by a meta-character
('*', '?', etc.). This can be achieved through a negative look ahead regular expression (regex
= "\\\\(?!(\\*|\\?))"). 
5. Use '^' as an escape character on the Windows platform
 - On the negative side, this model would push the platform differences to platforms built
on top of Hadoop instead of keeping them consistent. Hadoop documentation broadly available
could still get someone to make a mistake described in HADOOP-8139. Just adding this for completeness,
I don’t think we want to go this route.

#3 seems like a nice long term solution. We would have to do a deeper dive on what would it
take to do it/prototyping, but seems like a good proposal from high-level. 

> globStatus using backslash for escaping does not work on Windows
> ----------------------------------------------------------------
>                 Key: HDFS-4632
>                 URL: https://issues.apache.org/jira/browse/HDFS-4632
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 2.1.1-beta
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HDFS-4632-trunk.patch
> {{Path}} normalizes backslashes to forward slashes on Windows.  Later, when passed to
{{FileSystem#globStatus}}, the path is no longer treated as an escape sequence.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message