hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Fri, 15 Aug 2014 22:07:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14099243#comment-14099243
] 

Sanjay Radia commented on HDFS-6134:
------------------------------------

We have made very good progress over the last few days. Thanks for taking the time for the
offline technical discussions.  Below is a  summary of   the concerns I have raised previously
in this Jira.
# Fix distcp and cp to *automatically* deal with EZ  using /r/r internally. Initially   we
 need to support only  row 1 and row 4 in the table I attached  in Hadoop-10919
# Fix Webhdfs to use KMS delegation tokens so that webhdfs can be used with transparent encryption
 without giving user "hdfs" KMS proxy permission (and as a result to admins). Rest is a key
 protocol for HDFS and for many Hadoop use cases, an Admin should not have access to the keys
of  encrypted files.
# Further work on specifying what HAR should do (I have listed some use cases and proposed
solutions ), and then follow it up with a fix to har.
# Some work on understanding availability and scalability on KMS for medium to large clusters.
Perhaps we need to explore getting the keys ahead of time when a job is submitted.

Lets complete Items 1 and 2 promptly. Before we publish transparent encryption in a   2.x
 release for pubic consumption, let us at  least complete item 1 (ie distcp and cp) and the
flag to turn this feature on/of.



> Transparent data at rest encryption
> -----------------------------------
>
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0, 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Charles Lamb
>         Attachments: HDFS-6134.001.patch, HDFS-6134.002.patch, HDFS-6134_test_plan.pdf,
HDFSDataatRestEncryption.pdf, HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
>
>
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
or WebHDFS REST API.
> The resulting implementation should be able to be used in compliance with different regulation
requirements.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message