spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjia Wang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-11413) Java 8 build has problem with joda-time and s3 request, should bump joda-time version
Date Fri, 30 Oct 2015 16:03:27 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982755#comment-14982755
] 

Yongjia Wang edited comment on SPARK-11413 at 10/30/15 4:02 PM:
----------------------------------------------------------------

My last statement was wrong.
It's a problem when using jre 8u60 or above, and when joda-time 2.8.0 or below was loaded
at run time.
The following error was reproduced when I use the official spark release and run with JRE
1.8.0_65 (Oracle Corporation)
Note that the aws-java-sdk-s3 library already depends on joda-time-2.8.1. It's due to Spark
assembly's inclusion of joda-time-2.5 which was loaded by JVM that caused this problem.

com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date
or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request
ID: E0D714AF923221DD), S3 Extended Request ID: 8OJlBMYKVzwrEXHANpBTsiLRmaFn2lHQ4sAvkU/HF66wWGVj8VR2/Jh4Wl8QuFUt
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3738)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:653)


was (Author: yongjiaw):
My last statement was wrong.
It's a problem when using jre 8u60 or above, and when joda-time 2.8.0 or below was loaded
at run time.
The following error was reproduced when I use the official spark release with JRE 1.8.0_65
(Oracle Corporation)
Note that the aws-java-sdk-s3 library already depends on joda-time-2.8.1. It's due to Spark
assembly's inclusion of joda-time-2.5 which was loaded by JVM that caused this problem.

com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date
or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request
ID: E0D714AF923221DD), S3 Extended Request ID: 8OJlBMYKVzwrEXHANpBTsiLRmaFn2lHQ4sAvkU/HF66wWGVj8VR2/Jh4Wl8QuFUt
	at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3738)
	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:653)

> Java 8 build has problem with joda-time and s3 request, should bump joda-time version
> -------------------------------------------------------------------------------------
>
>                 Key: SPARK-11413
>                 URL: https://issues.apache.org/jira/browse/SPARK-11413
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>            Reporter: Yongjia Wang
>            Priority: Minor
>
> Joda-time has problems with formatting time zones starting with Java 1.8u60, and this
will cause s3 request to fail. It is said to have been fixed at joda-time 2.8.1.
> Spark is still using joda-time 2.5 by fault, if java8 is used to build spark, should
set -Djoda.version=2.8.1 or above.
> I was hit by this problem, and -Djoda.version=2.9 worked.
> I don't see any reason not to bump up joda-time version in pom.xml
> Should I create a pull request for this? It is trivial.
> https://github.com/aws/aws-sdk-java/issues/484 
> https://github.com/aws/aws-sdk-java/issues/444
> http://stackoverflow.com/questions/32058431/aws-java-sdk-aws-authentication-requires-a-valid-date-or-x-amz-date-header



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message