hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Igor Mazur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13887) Support for client-side encryption in S3A file system
Date Thu, 04 May 2017 10:25:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996504#comment-15996504
] 

Igor Mazur commented on HADOOP-13887:
-------------------------------------

As I understand, the main problem is that FSInputStream implements Seekable and PositionedReadable
by default. So every other code was written from this assumption.
Can't evaluate is it a good assumption or not, for all cases :) But looks like making this
part more flexible - is an enormous amount of coding and testing.

So, maybe we need to try another approach - return file as-is from S3 but also return metadata,
that includes a type of encryption and encrypted CEK and decrypt the file on higher layers.
 I see classes with names CryptoInputStream, etc. Haven't looked how they work yet - just
idea. 
The biggest problem with this approach - is a duplication of encryption/decryption logic from
AWS SDK. Looks like it will be hard to reuse same classes from SDK - because encryption/decryption
tightly linked with getting/putting objects from S3 there.


> Support for client-side encryption in S3A file system
> -----------------------------------------------------
>
>                 Key: HADOOP-13887
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13887
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Jeeyoung Kim
>            Assignee: Igor Mazur
>            Priority: Minor
>         Attachments: HADOOP-13887-002.patch, HADOOP-13887-007.patch, HADOOP-13887-branch-2-003.patch,
HADOOP-13897-branch-2-004.patch, HADOOP-13897-branch-2-005.patch, HADOOP-13897-branch-2-006.patch,
HADOOP-13897-branch-2-008.patch, HADOOP-13897-branch-2-009.patch, HADOOP-13897-branch-2-010.patch,
HADOOP-13897-branch-2-012.patch, HADOOP-13897-branch-2-014.patch, HADOOP-13897-trunk-011.patch,
HADOOP-13897-trunk-013.patch, HADOOP-14171-001.patch
>
>
> Expose the client-side encryption option documented in Amazon S3 documentation  - http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
> Currently this is not exposed in Hadoop but it is exposed as an option in AWS Java SDK,
which Hadoop currently includes. It should be trivial to propagate this as a parameter passed
to the S3client used in S3AFileSystem.java



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message