Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 15 Jun 2017 13:22:00 +0000 (UTC)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13080086.1497532790000.24594.1497532920205@Atlassian.JIRA>
In-Reply-To: <JIRA.13080086.1497532790000@Atlassian.JIRA>
References: <JIRA.13080086.1497532790000@Atlassian.JIRA> <JIRA.13080086.1497532790578@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HADOOP-14528) s3a encryption tests fail when
 dest bucket has fs.s3a.server-side-encryption.key  set
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 15 Jun 2017 13:22:06 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-14528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050464#comment-16050464 ] 

Steve Loughran commented on HADOOP-14528:
-----------------------------------------

Stack in SSE-C test. AWS s3 is handed the key ref from {{fs.s3a.server-side-encryption.key}}, assuming it's a base-64 key and failing at parse time.

This is something to consider covering in troubleshooting docs.

{code}

Running org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSECBlockOutputStream
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.156 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSECBlockOutputStream
testEncryptionOverRename(org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSECBlockOutputStream)  Time elapsed: 0.042 sec  <<< ERROR!
java.lang.IllegalArgumentException: Input is expected to be encoded in multiple of 4 bytes but found: 75
	at com.amazonaws.util.Base64Codec.decode(Base64Codec.java:198)
	at com.amazonaws.util.Base64.decode(Base64.java:89)
	at com.amazonaws.services.s3.AmazonS3Client.populateSSE_C(AmazonS3Client.java:3840)
	at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1247)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:941)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1611)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerMkdirs(S3AFileSystem.java:1562)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:1539)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2230)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.mkdirs(AbstractFSContractTestBase.java:338)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:193)
	at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

{code}

> s3a encryption tests fail when dest bucket has fs.s3a.server-side-encryption.key  set
> -------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14528
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14528
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>         Environment: test bucket with SSE-KMS required
>            Reporter: Steve Loughran
>
> When testing against a bucket set up to require SSE-KMS, and with the bucket settings enabling this & providing the key in {{ fs.s3a.server-side-encryption.key}}, some of the encryption tests fail.
> Not sure whether this can/should be fixed, except by saying "disable encryption tests here" that is: don't try to be clever about detecting this conditions and skipping the tests automatically


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org