hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14528) s3a encryption tests fail when dest bucket has fs.s3a.server-side-encryption.key set
Date Thu, 15 Jun 2017 13:22:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050464#comment-16050464
] 

Steve Loughran commented on HADOOP-14528:
-----------------------------------------

Stack in SSE-C test. AWS s3 is handed the key ref from {{fs.s3a.server-side-encryption.key}},
assuming it's a base-64 key and failing at parse time.

This is something to consider covering in troubleshooting docs.

{code}

Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSECBlockOutputStream
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.156 sec <<< FAILURE!
- in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSECBlockOutputStream
testEncryptionOverRename(org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSECBlockOutputStream)
 Time elapsed: 0.042 sec  <<< ERROR!
java.lang.IllegalArgumentException: Input is expected to be encoded in multiple of 4 bytes
but found: 75
	at com.amazonaws.util.Base64Codec.decode(Base64Codec.java:198)
	at com.amazonaws.util.Base64.decode(Base64.java:89)
	at com.amazonaws.services.s3.AmazonS3Client.populateSSE_C(AmazonS3Client.java:3840)
	at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1247)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:941)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1611)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerMkdirs(S3AFileSystem.java:1562)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:1539)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2230)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:338)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:193)
	at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

{code}

> s3a encryption tests fail when dest bucket has fs.s3a.server-side-encryption.key  set
> -------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14528
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14528
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>         Environment: test bucket with SSE-KMS required
>            Reporter: Steve Loughran
>
> When testing against a bucket set up to require SSE-KMS, and with the bucket settings
enabling this & providing the key in {{ fs.s3a.server-side-encryption.key}}, some of the
encryption tests fail.
> Not sure whether this can/should be fixed, except by saying "disable encryption tests
here" that is: don't try to be clever about detecting this conditions and skipping the tests
automatically



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message