hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14143) S3A Path Style Being Ignore
Date Sun, 05 Mar 2017 12:56:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896248#comment-15896248
] 

Steve Loughran commented on HADOOP-14143:
-----------------------------------------

swell, we're setting it in the builder on the AWS SDK. I'll leave it to you debug what's going
on. Try with branch-2 and the updated AWS SDK, in case amazon have fixed the issue.

> S3A Path Style Being Ignore
> ---------------------------
>
>                 Key: HADOOP-14143
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14143
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 2.7.3
>            Reporter: Vishnu Vardhan
>            Priority: Minor
>
> Hi:
> In the following example, path style specification is being ignore
> scala>:paste
> sc.setLogLevel("DEBUG")
> sc.hadoopConfiguration.set("fs.s3a.impl","org.apache.hadoop.fs.s3a.S3AFileSystem")
> sc.hadoopConfiguration.set("fs.s3a.endpoint","webscaledemo.netapp.com:8082")
> sc.hadoopConfiguration.set("fs.s3a.access.key","")
> sc.hadoopConfiguration.set("fs.s3a.secret.key","")
> sc.hadoopConfiguration.set("fs.s3a.path.style.access","false")
> val s3Rdd = sc.textFile("s3a://myBkt8")
> s3Rdd.count()
> Debug Log:
> application/x-www-form-urlencoded; charset=utf-8
> Thu, 02 Mar 2017 22:46:56 GMT
> /myBkt8/"
> 17/03/02 14:46:56 DEBUG request: Sending Request: GET https://webscaledemo.netapp.com:8082
/myBkt8/ Parameters: (max-keys: 1, prefix: user/vardhan/, delimiter: /, ) Headers: (Authorization:
AWS 2SNAJYEMQU45YPVYC89D:PIQqLcr6FV61H0+Ay7tw3WygGFo=, User-Agent: aws-sdk-java/1.7.4 Mac_OS_X/10.12.3
Java_HotSpot(TM)_64-Bit_Server_VM/25.60-b23/1.8.0_60, Date: Thu, 02 Mar 2017 22:46:56 GMT,
Content-Type: application/x-www-form-urlencoded; charset=utf-8, ) 
> 17/03/02 14:46:56 DEBUG PoolingClientConnectionManager: Connection request: [route: {s}->https://webscaledemo.netapp.com:8082][total
kept alive: 0; route allocated: 0 of 15; total allocated: 0 of 15]
> 17/03/02 14:46:56 DEBUG PoolingClientConnectionManager: Connection leased: [id: 2][route:
{s}->https://webscaledemo.netapp.com:8082][total kept alive: 0; route allocated: 1 of 15;
total allocated: 1 of 15]
> 17/03/02 14:46:56 DEBUG DefaultClientConnectionOperator: Connecting to webscaledemo.netapp.com:8082
> 17/03/02 14:46:57 DEBUG RequestAddCookies: CookieSpec selected: default
> 17/03/02 14:46:57 DEBUG RequestAuthCache: Auth cache not set in the context
> 17/03/02 14:46:57 DEBUG RequestProxyAuthentication: Proxy auth state: UNCHALLENGED
> 17/03/02 14:46:57 DEBUG SdkHttpClient: Attempt



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message