Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 23 Mar 2017 13:18:42 +0000 (UTC)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13027140.1481327284000.101036.1490275122013@Atlassian.JIRA>
In-Reply-To: <JIRA.13027140.1481327284000@Atlassian.JIRA>
References: <JIRA.13027140.1481327284000@Atlassian.JIRA> <JIRA.13027140.1481327284222@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HADOOP-13887) Support for client-side
 encryption in S3A file system
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 23 Mar 2017 13:18:50 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15938268#comment-15938268 ] 

Steve Loughran commented on HADOOP-13887:
-----------------------------------------

I like how this is coming together; it's always good to keep checkstyle quiet, even though it complains too much for my personal liking.

On the change to {{writeDataset();}}, how about retaining a method with the original signature, and calling the new one with the last arg set to true? That way: fewer changes to the codebase, anything downstream using CTU (I'm the likeliest culprit) doesn't break.

Other than that, I don't see any more code changes. Any other reviewers want to add. Anyone tested it yet?

I'm thinking of end user docs. That's something we could just collaborate on in the comments here, rather than iterate through the code patches, which are pretty stable to me. As well as some instructions on what to do, and warnings, it'd be good to have a bit in the troubleshooting section. I can see various problems arising: 

* encryption enabled, no key
* encryption enabled, wrong key
* encryption enabled, no JCE That crops in kerberos BTW; the KDiag entry point explicitly tests for it. We could say "use kdiag to look for that".
* encryption enabled, no bouncy castle.
* encryption enabled, object store doesn't support it
* encryption disabled, end data encrypted.

It'd be good to have whatever stack traces you've managed to collect as part of this, otherwise we can make some more; easily done :)


> Support for client-side encryption in S3A file system
> -----------------------------------------------------
>
>                 Key: HADOOP-13887
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13887
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Jeeyoung Kim
>            Assignee: Igor Mazur
>            Priority: Minor
>         Attachments: HADOOP-13887-002.patch, HADOOP-13887-007.patch, HADOOP-13887-branch-2-003.patch, HADOOP-13897-branch-2-004.patch, HADOOP-13897-branch-2-005.patch, HADOOP-13897-branch-2-006.patch, HADOOP-13897-branch-2-008.patch, HADOOP-14171-001.patch
>
>
> Expose the client-side encryption option documented in Amazon S3 documentation  - http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
> Currently this is not exposed in Hadoop but it is exposed as an option in AWS Java SDK, which Hadoop currently includes. It should be trivial to propagate this as a parameter passed to the S3client used in S3AFileSystem.java


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org