hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8139) Path does not allow metachars to be escaped
Date Fri, 09 Mar 2012 20:38:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226429#comment-13226429

Daryn Sharp commented on HADOOP-8139:

I was rushing to meeting, so allow me to re-summarize what (I think) I did.
* Existing Path ctors require the path to have a dir sep of / because they expected to be
* Path(File) can instantiate path objects for the local fs.  For unix, it does the same as
Path(file.toString()), so no change.  For windows, it allows \ paths by rewriting the File
to sub \ to /.
* RLFS will rewrite windows paths to sub ^ with \ in order to preserve windows quoting semantics.
 Unix is unchanged.

This means that windows users can use \ paths only with {{File}} objects, and convert to {{Path}}
with {{new Path(new File("\foo\bar"))}} which turns into {{/foo/bar}}.  However, {{new Path("\foo\bar")}}
is presumed to be a URI so it remains {{\foo\bar}}.  RLFS for windows will allow {{^*}} to
be rewritten into {{\ *}} so the normal globbing code will operate as expected.

In short:
# Behavior is consistent between unix and windows, unless windows uses File objects to obtain
local semantics
# Unix allows \ as a meta-quote.
# Windows RLFS paths may use \ and ^ semantics, while other paths follow the URI semantics
of / and \.

> Path does not allow metachars to be escaped
> -------------------------------------------
>                 Key: HADOOP-8139
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8139
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-8139-2.patch, HADOOP-8139-3.patch, HADOOP-8139.patch, HADOOP-8139.patch
> Path converts "\" into "/", probably for windows support?  This means it's impossible
for the user to escape metachars in a path name.  Glob expansion can have deadly results.
> Here are the most egregious examples. A user accidentally creates a path like "/user/me/*/file".
 Now they want to remove it.
> {noformat}"hadoop fs -rmr -skipTrash '/user/me/\*'" becomes...
> "hadoop fs -rmr -skipTrash /user/me/*"{noformat}
> * User/Admin: Nuked their home directory or any given directory
> {noformat}"hadoop fs -rmr -skipTrash '\*'" becomes...
> "hadoop fs -rmr -skipTrash /*"{noformat}
> * User:  Deleted _everything_ they have access to on the cluster
> * Admin: *Nukes the entire cluster*
> Note: FsShell is shown for illustrative purposes, however the problem is in the Path
object, not FsShell.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message