hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13887) Support for client-side encryption in S3A file system
Date Tue, 28 Mar 2017 11:04:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944947#comment-15944947

Steve Loughran commented on HADOOP-13887:

the stack traces should go into the troubleshooting section in index.md, or maybe we could
add a whole new page on encryption?

Patch minor nits

h3. {{S3ClientFactory}}

Whatever your IDE is doing to reformatting the copyright header and the imports here, it should
stop it. Imports are a very brittle part of the merge point of patches, and need to be touched
very carefully. no reordering, no .* except on static imports, etc. 

I don't like the exception logic in {{getAmazonS3EncryptionClient}}; it' raising an IllegalArgumentException,
then catching and rethrowing. 

Here's what I prefer

* use guava {{Preconditions}} to check conditions like the key ID
* Don't catch and wrap exceptions at this point, except when adding extra diagnostics. I can
see that you are copying some bits of the existing codebsase, but that doesn't mean we got
that right earlier.

I'' have to review the other stuff more carefully, by applying the patch and actually using
it. I won't have time for that until mid April, and I'd like to get HADOOP-13345 preview merged
in; it changes the client factory enough (subclassing) that this patch won't apply. That code
has a head start on merging in, i'm afraid.

> Support for client-side encryption in S3A file system
> -----------------------------------------------------
>                 Key: HADOOP-13887
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13887
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Jeeyoung Kim
>            Assignee: Igor Mazur
>            Priority: Minor
>         Attachments: HADOOP-13887-002.patch, HADOOP-13887-007.patch, HADOOP-13887-branch-2-003.patch,
HADOOP-13897-branch-2-004.patch, HADOOP-13897-branch-2-005.patch, HADOOP-13897-branch-2-006.patch,
HADOOP-13897-branch-2-008.patch, HADOOP-13897-branch-2-009.patch, HADOOP-13897-branch-2-010.patch,
HADOOP-13897-branch-2-012.patch, HADOOP-13897-branch-2-014.patch, HADOOP-13897-trunk-011.patch,
HADOOP-13897-trunk-013.patch, HADOOP-14171-001.patch
> Expose the client-side encryption option documented in Amazon S3 documentation  - http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
> Currently this is not exposed in Hadoop but it is exposed as an option in AWS Java SDK,
which Hadoop currently includes. It should be trivial to propagate this as a parameter passed
to the S3client used in S3AFileSystem.java

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message