hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-15314) Scheme assertion in DynamoDBMetadataStore::checkPath is unnecessarily restrictive
Date Fri, 16 Mar 2018 14:04:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated HADOOP-15314:
    Issue Type: Sub-task  (was: Bug)
        Parent: HADOOP-15226

> Scheme assertion in DynamoDBMetadataStore::checkPath is unnecessarily restrictive
> ---------------------------------------------------------------------------------
>                 Key: HADOOP-15314
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15314
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: DJ Hoffman
>            Priority: Major
> In version 3.0.0, the checkPath method for dealing with paths prevents us from using
the s3:// scheme when utilizing S3Guard. However, in our core-site.xml we have included 
> {noformat}
>   <property>
>     <name>fs.s3.impl</name>
>     <value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
>   </property>{noformat}
> which should enforce that s3 prefixed paths go through s3a and are properly compatible
with s3guard. We removed the assertion that paths use the s3a scheme (some of our paths use
the s3 scheme) and our testing thus far with S3Guard enabled have been positive. We believe
the assertion in checkPath is unnecessary and could be expanded to include the s3 and s3n
schemes if not dropped altogether or altered in some other way. We're happy to develop and
test a patch if the community is amenable to the change.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message