hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15140) S3guard mistakes root URI without / as non-absolute path
Date Wed, 31 Jan 2018 01:25:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346095#comment-16346095

Steve Loughran commented on HADOOP-15140:

* Well, you've got the evidence that there's nothing wrong with s3guard or s3a, just that
it was there where I found it.
* and, with HDFS setting the policy, we are consistent.
* Most likely to fail here is bits of spark or hive which pass around URIs; you may get jobs
which fail if the test is the root of the bucket. Unfortunate, but, well, not so common a
use that we get lots of complaints.

Local FS is the odd one as it varies from wherever you run the tests (and the JVM), so we
can't use it as a reference for anything, and trying to make it consistent with HDFS in terms
of behaviour or error reporting not something you can ever complete.

I think we may want to consider this "at least consistent with HDFS", retitle it S3A (as it
is more than just s3guard), and mark as a WONTFIX, or at least pull in the test which works

> S3guard mistakes root URI without / as non-absolute path
> --------------------------------------------------------
>                 Key: HADOOP-15140
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15140
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Abraham Fine
>            Priority: Major
> If you call {{getFileStatus("s3a://bucket")}} then S3Guard will throw an exception in
putMetadata, as it mistakes the empty path for "non-absolute path"

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message