hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Thu, 19 Jun 2014 20:04:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037778#comment-14037778

Alejandro Abdelnur commented on HDFS-6134:

[~sanjay.radia], thanks for jumping in.

Things has changed a bit since the latest design doc based on the received feedback, mostly
from [~owen.omalley] and [~yoderme]. (I will update the design doc to reflect these changes).

On *distcp:*

Vanilla  distcp will just work with transparent encryption. Data will be decrypted on read
and encrypted on write, assuming both source and target are in encrypted zones. 

The proposal on changing distcp is to enable a second use used case, copy data from one cluster
to another without having to decrypt/encrypt the data while doing the copy. This is useful
when doing copies for disaster recovery, hdfs admins could do the copy without having to have
access to the encryption keys.

On *relying on kerberos credentials OR delegation tokens to obtain keys:*

It works exactly like HDFS. KMS will support both Kerberos and delegation tokens. A Kerberized
client can request a KMS delegation token which is serialized with the rest of the credentials
to be used by containers running in the cluster. It is assumed you are using network encryption
as well to avoid delegation tokens sniffing.

On *relying on file-ACLs to hand out keys:*

No, file-ACLs give you access to the data in HDFS. You also need to have access to the Key,
that is the responsibility of the KeyProvider to do.

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>         Attachments: HDFSDataAtRestEncryption.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message