hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Wed, 18 Jun 2014 23:06:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036617#comment-14036617
] 

Alejandro Abdelnur commented on HDFS-6134:
------------------------------------------

[~lmccay], thanks for following up.

The proposed approach would be applicable using different cipher modes than CTR (ie CBC and
XTR, even GCM if we handle the offset correction changes because of the GCM tag). In all cases,
it would enable keeping the EZ keyVersion materials unknown to HDFS and clients and exposing
the DEK for the files being accessed to the client accessing the file only. 

In the case of CTR, the proposed approach also helps avoiding the IV reuse scenario.

Trying to answer your questions:

bq. how does the keyprovider know what EZ key to use - is it the key that is referenced by
the keyVersionName?

{code}
  // creates EDEK using specified version name
  KeyVersion generateEncryptedKey(String versionName, byte[] iv) 

  // receives EDEK returns DEK using specified version name
  KeyVersion decryptEncryptedKey(String versionName, byte[] iv, KeyVersion encryptedKey) 
{code}

The callers of both methods have the versionName at hand

bq. how do we key HDFS clients from asking for the EZ key - if it is stored by the passed
in keyVersionName?

The file iNode will store (EZ-keyVersionName, IV, EDEK), that info is passed to the client
on create()/open(). Using that info, the client can go to the KeyProvider to get the DEK for
the file.

bq. will this require special access control protection for EZ keys?

KeyProviders could implement special control. For example KMS allows, via ACLs, to get KeyVersion
without key material. This will effectively prevent EZ keys from leaving the KMS.

bq. would the unique DEK be stored in the provider as well or only in the extended attributes
of the file? if stored in the provider what is the keyVersionName for it?

the unique EDEKs (encrypted with the EZ keyversion) are not stored by the KeyProvider but
in the xAttr of the file.


> Transparent data at rest encryption
> -----------------------------------
>
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>         Attachments: HDFSDataAtRestEncryption.pdf
>
>
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
or WebHDFS REST API.
> The resulting implementation should be able to be used in compliance with different regulation
requirements.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message