Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Thu, 8 Mar 2012 23:05:58 +0000 (UTC)
From: "Daryn Sharp (Commented) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <1584111203.41108.1331247958622.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <393232168.22994.1330968477181.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-8139) Path does not allow metachars to
 be escaped
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225647#comment-13225647 ] 

Daryn Sharp commented on HADOOP-8139:
-------------------------------------

I'm sorry,  I misunderstood what I was expected to do.  We'd like to get this into 23.2 so I've been trying to avoid an in-depth change.  Adding a windows blocker is undesirable too.

Apparently windows NT has long internally handled / as \, plus the JVM will implicitly do the conversion.  I read it on the internet, so it must be true!  However, for File objects returned from operations, I'm not sure if it's smart enough to use the same delimiter used in the input path.

Here's some options.  Doug, I think you are essentially proposing #2?
# Add a config option to sub \ to /
#* Pros: backward-compatibility for those that want it (ie. no quoting), or Unix and windows can quote.
#* Cons: none?
# No subbing of \ to / (latest patch)
#* Pros: Unix and windows can quote metas
#* Cons: File objects returned from Windows might not convert to Path.  Requires changes to RLFS (*)
# Use the WINDOWS conditional for subbing \ to /
#* Pros: Unix can quote metas.
#* Cons: Unix can't use windows-style paths to access hdfs. Windows can't quota metas.  Not a regression because it can't do that today, but now inconsistent with unix feature.
# Use a WINDOWS conditional for subbing \ to /, and ^ to \ (prior patch)
#* Pros: Unix and windows can quota metas
#* Cons: The quoting requires different/inconsistent chars that are system-dependent.  Unix can't use windows-style paths.

(*) A new Path ctor might make this easy to fix:
{code}
Path(File file) {
  (File.separatorChar == Path.SEPARATOR_CHAR)
    ? this(file.toString())
    : this(file.toString().replace(File.separatorChar, Path.SEPARATOR_CHAR));
  }
}
{code}
                
> Path does not allow metachars to be escaped
> -------------------------------------------
>
>                 Key: HADOOP-8139
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8139
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-8139-2.patch, HADOOP-8139.patch, HADOOP-8139.patch
>
>
> Path converts "\" into "/", probably for windows support?  This means it's impossible for the user to escape metachars in a path name.  Glob expansion can have deadly results.
> Here are the most egregious examples. A user accidentally creates a path like "/user/me/*/file".  Now they want to remove it.
> {noformat}"hadoop fs -rmr -skipTrash '/user/me/\*'" becomes...
> "hadoop fs -rmr -skipTrash /user/me/*"{noformat}
> * User/Admin: Nuked their home directory or any given directory
> {noformat}"hadoop fs -rmr -skipTrash '\*'" becomes...
> "hadoop fs -rmr -skipTrash /*"{noformat}
> * User:  Deleted _everything_ they have access to on the cluster
> * Admin: *Nukes the entire cluster*
> Note: FsShell is shown for illustrative purposes, however the problem is in the Path object, not FsShell.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira