hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Wed, 13 Aug 2014 16:26:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095668#comment-14095668

Alejandro Abdelnur commented on HDFS-6134:

Let me try to explain things a different way.

When setting up filesystem encryption in HDFS (forget about webhdfs and httpfs for now), things
will be configured so the HDFS superuser cannot retrieve decrypted 'file encryption keys'.
Because the HDFS superuser has access to the encrypted versions of the files, having access
to the decrypted 'file encryption keys' would allow the HDFS superuser to get access to the
decrypted file. One of the goals of HDFS encryption is to prevent that.

This is achieved by blacklisting the HDFS superuser from retrieving decrypted 'file encryption
keys' from the KMS. This blacklist is must be enforced on the real UGI hitting the KMS (regardless
if it is doing a doAs or not).

If you set up httpfs, it runs using the 'httpfs' user, a HDFS regular user configured as proxyuser
to interact with HDFS and KMS doing doAs calls. 

If you set up webhdfs, it runs using the 'hdfs' user, the HDFS superuser, and this user will
have to be configured as proxyuser in the KMS to work with doAs calls. Also the 'hdfs' user
will have to be removed from the KMS decrypt-keys blacklist (*and this is the problem*).

Even if you audit the webhdfs code running in the DNs to ensure things are always done using
doAs and that there is no foul play in the DN code there is an issue. The issue is:

* An HDFS admin logins to a DN in the cluster as 'hdfs'
* Then he kinits as 'hdsf/HOST'
* Then he curls the KMS asking to decrypted keys as user X doing  a doAs
* Because he has access to the encrypted file, and now has the decrypted key, gets access
to the file in clear

hope this clarifies.

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0, 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Charles Lamb
>         Attachments: HDFS-6134.001.patch, HDFS-6134.002.patch, HDFS-6134_test_plan.pdf,
HDFSDataatRestEncryption.pdf, HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message