hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Igor Mazur (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-13887) Support for client-side encryption in S3A file system
Date Tue, 28 Mar 2017 10:43:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944922#comment-15944922
] 

Igor Mazur edited comment on HADOOP-13887 at 3/28/17 10:43 AM:
---------------------------------------------------------------

Documentation related:

2) No BouncyCastle jar in classpath:
{noformat}
java.lang.IllegalArgumentException: Error during initialization of AmazonS3EncryptionClient
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.getAmazonS3EncryptionClient(S3ClientFactory.java:275)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createAmazonS3Client(S3ClientFactory.java:208)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createS3Client(S3ClientFactory.java:80)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:189)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3241)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:467)
	at org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Caused by: java.lang.UnsupportedOperationException: The Bouncy castle library jar is required
on the classpath to enable authenticated encryption
	at com.amazonaws.services.s3.model.CryptoConfiguration.check(CryptoConfiguration.java:241)
	at com.amazonaws.services.s3.model.CryptoConfiguration.<init>(CryptoConfiguration.java:72)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.getAmazonS3EncryptionClient(S3ClientFactory.java:256)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createAmazonS3Client(S3ClientFactory.java:208)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createS3Client(S3ClientFactory.java:80)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:189)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3241)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:467)
	at org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}


was (Author: igor mazur):
Documentation related:

2) No BouncyCastle jar in classpath:

java.lang.IllegalArgumentException: Error during initialization of AmazonS3EncryptionClient
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.getAmazonS3EncryptionClient(S3ClientFactory.java:275)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createAmazonS3Client(S3ClientFactory.java:208)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createS3Client(S3ClientFactory.java:80)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:189)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3241)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:467)
	at org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Caused by: java.lang.UnsupportedOperationException: The Bouncy castle library jar is required
on the classpath to enable authenticated encryption
	at com.amazonaws.services.s3.model.CryptoConfiguration.check(CryptoConfiguration.java:241)
	at com.amazonaws.services.s3.model.CryptoConfiguration.<init>(CryptoConfiguration.java:72)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.getAmazonS3EncryptionClient(S3ClientFactory.java:256)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createAmazonS3Client(S3ClientFactory.java:208)
	at org.apache.hadoop.fs.s3a.S3ClientFactory$DefaultS3ClientFactory.createS3Client(S3ClientFactory.java:80)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:189)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3241)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:467)
	at org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72)
	at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

> Support for client-side encryption in S3A file system
> -----------------------------------------------------
>
>                 Key: HADOOP-13887
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13887
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Jeeyoung Kim
>            Assignee: Igor Mazur
>            Priority: Minor
>         Attachments: HADOOP-13887-002.patch, HADOOP-13887-007.patch, HADOOP-13887-branch-2-003.patch,
HADOOP-13897-branch-2-004.patch, HADOOP-13897-branch-2-005.patch, HADOOP-13897-branch-2-006.patch,
HADOOP-13897-branch-2-008.patch, HADOOP-13897-branch-2-009.patch, HADOOP-13897-branch-2-010.patch,
HADOOP-13897-branch-2-012.patch, HADOOP-13897-trunk-011.patch, HADOOP-14171-001.patch
>
>
> Expose the client-side encryption option documented in Amazon S3 documentation  - http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
> Currently this is not exposed in Hadoop but it is exposed as an option in AWS Java SDK,
which Hadoop currently includes. It should be trivial to propagate this as a parameter passed
to the S3client used in S3AFileSystem.java



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message