hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14291) S3a "Bad Request" message to include diagnostics
Date Thu, 27 Apr 2017 10:49:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986342#comment-15986342

Steve Loughran commented on HADOOP-14291:

+ proxies, IP Address of proxies and S3 endpoint
+ if bucket requires encryption and client doesn't have it turned on, that also fails. Something
which can be probed

Maybe the strategy would be to bootstrap comms to the endpoint: lookup, remote server time
(in HTTP responses?), then 
* List buckets & see if the user is down as an owner for the target bucket (warn but don't
fail if not, as it may be someone else's public one, one in same org,...)
* attempt to read bucket root / 
* pick a file & do a HEAD
* try to read the file
* if enabled, attempt to write then delete an entry (some UUID on root; avoids directory creation

> S3a "Bad Request" message to include diagnostics
> ------------------------------------------------
>                 Key: HADOOP-14291
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14291
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
> There's a whole section in s3a troubleshooting because requests can get auth failures
for many reasons, including
> * no credentials
> * wrong credentials
> * right credentials, wrong bucket
> * wrong endpoint for v4 auth
> * trying to use private S3 server without specifying endpoint, so AWS being hit
> * clock out
> * joda time
> ....
> We can aid with debugging this by including as much as we can in in the message and a
URL To a new S3A bad auth wiki page.
> Info we could include
> * bucket
> * fs.s3a.endpoint
> * nslookup of endpoint
> * Anything else relevant but not a security risk
> Goal; people stand a chance of working out what is failing within a bounded time period

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message