hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14439) regression: secret stripping from S3x URIs breaks some downstream code
Date Tue, 05 Sep 2017 14:06:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated HADOOP-14439:
    Release Note: We have restored the retention of a username:secret in an s3a: URL if you
authenticate with S3 using embedded URI secrets. This is because some applications using strings
to marshall the URLs were breaking. However, using secrets in this way is dangerous as it
will end up in logs. It will be unsupported in Hadoop 3. To use different credentials in different
buckets, move to per-bucket configuration

> regression: secret stripping from S3x URIs breaks some downstream code
> ----------------------------------------------------------------------
>                 Key: HADOOP-14439
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14439
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 2.8.0
>         Environment: Spark 2.1
>            Reporter: Steve Loughran
>            Assignee: Vinayakumar B
>            Priority: Minor
>         Attachments: HADOOP-14439-01.patch, HADOOP-14439-02.patch
> Surfaced in SPARK-20799
> Spark is listing the contents of a path with getFileStatus(path), then looking up the
path value doing a lookup of the contents.
> Apparently the lookup is failing to find files if you have a secret in the key, {{s3a://key:secret@bucket/path}}.

> Presumably this is because the stripped values aren't matching.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message