hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Moist (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15006) Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
Date Mon, 29 Jan 2018 20:40:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343975#comment-16343975

Steve Moist commented on HADOOP-15006:

>what's your proposal for letting the client encryption be an optional feature, with key?

If s3a.client.encryption.enabled=true then check for BEZ if exists encrypt objects, else no
encryption for the bucket.  Or if the BEZI provider is configured as well rather than just
the flag.

>Is the file length as returned in listings 100% consistent with the amount of data you
get to read?


>I'm not going to touch this right now as its at the too raw stage

That's why I submitted it, for you and everyone else to play with to evaluate if this is something
that we should move forward with.  If needed I can go fix the broken S3Guard/Committer/byte
comparison tests and have yetus pass it, but the actual code is going to be about the same.


> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
> ---------------------------------------------------------------
>                 Key: HADOOP-15006
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15006
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs/s3, kms
>            Reporter: Steve Moist
>            Priority: Minor
>         Attachments: S3-CSE Proposal.pdf, s3-cse-poc.patch
> This is for the proposal to introduce Client Side Encryption to S3 in such a way that
it can leverage HDFS transparent encryption, use the Hadoop KMS to manage keys, use the `hdfs
crypto` command line tools to manage encryption zones in the cloud, and enable distcp to copy
from HDFS to S3 (and vice-versa) with data still encrypted.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message