hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15409) S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2
Date Fri, 27 Jul 2018 21:40:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560366#comment-16560366
] 

Steve Loughran commented on HADOOP-15409:
-----------------------------------------

Looking into the AWS SDK here, the v2 call does getBucketAcl(); if the user doesn't have the
perms to query the permissions it is still treated as proof of existence. The big diff here
is that permissions are checked early, which is good.

One thing I worry about: do all S3 re-implementations implement this call? Does anyone know
here?

> S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2
> ----------------------------------------------------------------
>
>                 Key: HADOOP-15409
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15409
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Priority: Blocker
>
> in S3AFileSystem.initialize(), we check for the bucket existing with verifyBucketExists(),
which calls s3.doesBucketExist(). But that doesn't check for auth issues. 
> s3. doesBucketExistV2() does at least validate credentials, and should be switched to.
This will help things fail faster 
> See SPARK-24000



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message