hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Mon, 23 Jun 2014 15:51:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040873#comment-14040873

Sanjay Radia commented on HDFS-6134:

Aaron said:
bq. distcp...   I disagree - this is exactly what one wants ..
So you are saying that distcp should decrypt and re-encrypt data as it copies it ... most
backup tools do not this as they copy data - it is extra CPU resources and further unneeded
venerability. There are customer use cases where  distcp not over an encrypted channel; hence
if one of the files being copied is encrypted one may not want the file to be transparently
sent decrypted. Further, a sensitive file in a subtree may have been encrypted because the
subtree is readable by a larger group and hence the distcp user may not have access to the

bq. delegation tokens - KMS ... Owen and Tucu have already discussed this quite a bit above
Turns out this issue come up in discussion with Owen, and he shares the concern and suggested
that I  post the concern. Besides even if Alejandro and Owen are in agreement, my question
is relevant and has not been raised so far above:  Encryption is used to overcome limitations
of authorization and authentication in the system. It is relevant to ask if the use of delegation
tokens to obtain keys adds weakness. 

bq. meeting ...
Aaron .. you are misunderstanding my point. I am not saying that the discussion on this jira
have not been open.
* See Alejandro's comments:  " Todd Lipcon and I had an offline discussion with Andrew Purtell,
Yi Liu and Avik Dey " and "After some offline discussions with Yi, Tianyou, ATM, Todd, Andrew
and Charles" ...
** there have been such meetings and I have *no objections* to  such private meetings because
I know that the bandwidth helps. I am merely asking for one more meeting where I can quickly
come up to speed on the context that Alejandro, Todd, Yi, Tianyou, Andrew, Atm,  share. It
will help me and others better understand the viewpoint that some of you share due to prevous
high bandwidth meetings.

**  There is a precedent of HDFS meetings in spite of open jira discussion - higher bandwidth
to progress faster.
**Perhaps I should have worded the "private meetings" differently ...  sorry if it came across
the wrong way.

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>         Attachments: HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message