hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kazuyuki Tanimura (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-14235) S3A Path does not understand colon (:) when globbing
Date Fri, 24 Mar 2017 23:32:42 GMT
Kazuyuki Tanimura created HADOOP-14235:

             Summary: S3A Path does not understand colon (:) when globbing
                 Key: HADOOP-14235
                 URL: https://issues.apache.org/jira/browse/HADOOP-14235
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/s3
    Affects Versions: 3.0.0-alpha2, 3.0.0-alpha1, 2.8.0, 2.8.1
            Reporter: Kazuyuki Tanimura

S3 paths, colons (:) are valid character in S3 paths. However, the Java URI class, which is
used in the Path class, does not allow it.

This becomes a problem particularly when we are globbing S3 paths. The globber thinks paths
with colons are invalid paths and throws URISyntaxException.

The reason is we are sharing Globber.java with all other Fs. Some of the rules for regular
Fs are not applicable to S3 just like this colon as an example.

Same issue is reported here https://issues.apache.org/jira/browse/SPARK-20061

The good news is I have a one line fix that I am about to send a pull request.

However, for a right fix, we should separate the S3 globber from the Globber.java as proposed
at https://issues.apache.org/jira/browse/HADOOP-13371

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message