Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 12 Dec 2017 22:39:00 +0000 (UTC)
From: "Steve Moist (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13115396.1509552442000.443266.1513118340204@Atlassian.JIRA>
In-Reply-To: <JIRA.13115396.1509552442000@Atlassian.JIRA>
References: <JIRA.13115396.1509552442000@Atlassian.JIRA> <JIRA.13115396.1509552442238@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HADOOP-15006) Encrypt S3A data client-side with
 Hadoop libraries & Hadoop KMS
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 12 Dec 2017 22:39:04 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288396#comment-16288396 ] 

Steve Moist commented on HADOOP-15006:
--------------------------------------

I don't think anyone's started it.  I posted the design doc in hopes of others looking at it and critiquing it in the background while I focus on other things, that once enough people had reviewed it, to start on it then.  The changes to the Hadoop CLI, KMS and other components was what worried me about it.  It's bigger in scope than just S3a. 

In the proposal I made, we didn't have an issue with the cipher text length and plaintext length as we used CTR with no padding vs the CBC with PKCS5Padding that the AWS sdk uses.  I wrote a quick prototype using AES/CTR/NoPadding and ran all the integration tests against it and it ran without issue and did diffs on the before/after of upload/download along with TerraSort and had no issues.

> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
> ---------------------------------------------------------------
>
>                 Key: HADOOP-15006
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15006
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs/s3, kms
>            Reporter: Steve Moist
>            Priority: Minor
>         Attachments: S3-CSE Proposal.pdf
>
>
> This is for the proposal to introduce Client Side Encryption to S3 in such a way that it can leverage HDFS transparent encryption, use the Hadoop KMS to manage keys, use the `hdfs crypto` command line tools to manage encryption zones in the cloud, and enable distcp to copy from HDFS to S3 (and vice-versa) with data still encrypted.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org