hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1995) Path can not handle a file name that contains a back slash
Date Thu, 04 Oct 2007 22:28:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532540

Doug Cutting commented on HADOOP-1995:

> I would vote that all paths are uris and thus must use "/" as the separator on all operating
systems and file systems.

That would certainly be nice, and we try to do that as much as possible.  Paths are always
normalized this way.  But if we start rejecting paths with backslashes, or interpreting backslashes
as quotations, Hadoop on windows will start exploding all over the place, with no easy central
place to fix things.

> I would push the flip from "/" to "\" in the local file system when running on windows.

As I mentioned above, not all paths come from a FileSystem impl so we can't depend on this
happening before we see a path, and folks process paths in os-independent code, traversing
directories, so delaying it until the filesystem sees the path won't work either.  I've tried
the high road, and it seems impassible.  There are also back-compatibility constraints: we
don't want to break user code, and a lot of user code processes paths.

I think cygwin is a good analogy.  Cygwin tries to use unix syntax and, at the same time,
support windows paths from, e.g., environment variables.  For the most part it works, but
there are a few edge cases where things don't work quite the same, as in the email I cited
above.  We need to minimize those edge cases to rare situations and have a ready workaround.
 But we may not be able to easily eliminate them.

You're welcome to try to try the high road yourself.  I've already spent more hours than I
care to trying to get Hadoop paths to work transparently across Windows and linux.  The current
solution is not arbitrary, but the result of lots of trial and error.

> Path can not handle a file name that contains a back slash
> ----------------------------------------------------------
>                 Key: HADOOP-1995
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1995
>             Project: Hadoop
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.14.1
>            Reporter: Hairong Kuang
>             Fix For: 0.16.0
> When normalizing a path name, Path incorrectly converts a back slash to a path separator
even if  the path name is of the unix style. This prohibs a glob from using a back slash to
escape a special character. A fix is to make path normalization file system dependent.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message