hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Larry McCay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13075) Add support for SSE-KMS and SSE-C in s3a filesystem
Date Thu, 26 Jan 2017 12:46:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839641#comment-15839641
] 

Larry McCay commented on HADOOP-13075:
--------------------------------------

At first glance, this method makes me a bit uneasy:

{noformat}
+   * Depending on which SSE encryption method is used, SSE-C uses the actual
 +   * key material to encrypt data.  This shouldn't be displayed.  If SSE-KMS is
 +   * enabled then it is a reference to the key id in AWS and is safe to display.
 +   * SSE-S3 is abstracted away and serverSideEncryptionKey would not be
 +   * populated.
 +   *
 +   * @return masked encryption key value, or the SSE KMS key id, or empty if
 +   * there is no encryption key
 +   */
 +  private String displayEncryptionKeyAs() {
 +    if(S3AEncryptionMethods.SSE_C.equals(serverSideEncryptionAlgorithm)) {
 +      return "************";
 +    } else if(S3AEncryptionMethods.SSE_KMS
 +        .equals(serverSideEncryptionAlgorithm)) {
 +      return S3AUtils.getServerSideEncryptionKey(getConf());
 +    }
 +    return "";
 +  }
{noformat}

It seems to be depending on a conf setting that could potentially be changed even if only
temporarily to get to the actual key material.
This sort of thing is generally better done in an object oriented manner.

I will try and dig into the S3AUtils.getPassword approach as well.
We may want to consider adding a getKey instead which would be based on the Key Provider API
rather than Credential Provider API and allow for versioning of the keys and the use of the
KMS server.
Maybe those things aren't important in the S3 usecase?


> Add support for SSE-KMS and SSE-C in s3a filesystem
> ---------------------------------------------------
>
>                 Key: HADOOP-13075
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13075
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Andrew Olson
>            Assignee: Federico Czerwinski
>
> S3 provides 3 types of server-side encryption [1],
> * SSE-S3 (Amazon S3-Managed Keys) [2]
> * SSE-KMS (AWS KMS-Managed Keys) [3]
> * SSE-C (Customer-Provided Keys) [4]
> Of which the S3AFileSystem in hadoop-aws only supports opting into SSE-S3 (HADOOP-10568)
-- the underlying aws-java-sdk makes that very simple [5]. With native support in aws-java-sdk
already available it should be fairly straightforward [6],[7] to support the other two types
of SSE with some additional fs.s3a configuration properties.
> [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html
> [2] http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html
> [3] http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html
> [4] http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html
> [5] http://docs.aws.amazon.com/AmazonS3/latest/dev/SSEUsingJavaSDK.html
> [6] http://docs.aws.amazon.com/AmazonS3/latest/dev/kms-using-sdks.html#kms-using-sdks-java
> [7] http://docs.aws.amazon.com/AmazonS3/latest/dev/sse-c-using-java-sdk.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message