hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8139) Path does not allow metachars to be escaped
Date Fri, 09 Mar 2012 01:05:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225755#comment-13225755

Daryn Sharp commented on HADOOP-8139:

bq. When would a non-windows user want subbing? When would a Windows user not require it?

I'm just presenting the options, but if we consider hadoop to be generally URI-based, then
no subbing should ever occur for URIs.  MS deprecated their non-standard use of \ in URIs
back in 2006.  A valid use case _may_ be C:\dir\file.  It's easy to identify the drive letter
and convert to file:///C:/dir/file -- official file URI per MS support docs.  However, once
you start using relative paths, it becomes murky whether \ is a dir sep or quoting char. 
If we want consistent paths and quoting that are system-independent, then we should mandate
/.  I think if we run external paths through File (at least for windows), it might normalize
for us, but I need to find myself a windows machine to test.

All said, I agree #2 is the ideal for consistency.  #4 or #3 is an immediate remedy for unix.
 #3 would not alter the existing windows behavior, giving us time to figure out how we want
to improve it.  I'm somewhat concerned that if we let the cat out the bag with a "^" on windows
(#4) we are going to have nasty cross-platform script compatibility in the future.

I'll investigate #2 tomorrow morning, but you've concerned me that it may be harder than it
looks, and I don't have the means to stomp down windows bugs.  What would you be willing to
accept in 23.2 if #2 proves difficult?

> Path does not allow metachars to be escaped
> -------------------------------------------
>                 Key: HADOOP-8139
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8139
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-8139-2.patch, HADOOP-8139.patch, HADOOP-8139.patch
> Path converts "\" into "/", probably for windows support?  This means it's impossible
for the user to escape metachars in a path name.  Glob expansion can have deadly results.
> Here are the most egregious examples. A user accidentally creates a path like "/user/me/*/file".
 Now they want to remove it.
> {noformat}"hadoop fs -rmr -skipTrash '/user/me/\*'" becomes...
> "hadoop fs -rmr -skipTrash /user/me/*"{noformat}
> * User/Admin: Nuked their home directory or any given directory
> {noformat}"hadoop fs -rmr -skipTrash '\*'" becomes...
> "hadoop fs -rmr -skipTrash /*"{noformat}
> * User:  Deleted _everything_ they have access to on the cluster
> * Admin: *Nukes the entire cluster*
> Note: FsShell is shown for illustrative purposes, however the problem is in the Path
object, not FsShell.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message