hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Lamb (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Thu, 14 Aug 2014 21:36:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097707#comment-14097707

Charles Lamb commented on HDFS-6134:

FileEncryptionInfo stores EZKeyID right along with EDEK. So using EZKeyID and EDEK one can
get at DEK right?

So I think Sanjay's assertion is right. When you want to create a file, you need encryption
zone's EZKey. For reading the FileEncruptionInfo is self sufficient. Am I missing something?

I might be misunderstanding what you're saying. You are correct that to create a file you
need the ez-keyid (actually you need the credentials to access the ez-key since the KMS is
the only one who touches the ezkey on the users behalf -- the actual ezkey never leaves the
KMS). The ezkey-id is required because you need to turn an EDEK into a DEK. But it is the
EDEK, not a DEK, that is stored in the xattr. After the file has been created (and encrypted),
to read it back, you need the EDEK to be decrypted and turned into a DEK. The client asks
the KMS to do that decryption using the EZ-key (the id of which is also stored in the FEInfo).
But the FEInfo does not have a DEK -- it has an EDEK and to do any encryption/decryption you
need the DEK. Remember that we never want to store a DEK in the xattr because that would be
giving the unencrypted key to the NN.

The FEInfo is sufficient as long as you still have the KMS around to do the decryption of
the EDEK into a DEK. You will always need the KMS to make that conversion and the KMS will
do that based on whatever ezkeyid you hand it.

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0, 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Charles Lamb
>         Attachments: HDFS-6134.001.patch, HDFS-6134.002.patch, HDFS-6134_test_plan.pdf,
HDFSDataatRestEncryption.pdf, HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message