hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Thu, 14 Aug 2014 05:24:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096590#comment-14096590

Alejandro Abdelnur commented on HDFS-6134:


HttpFS is a service that requires to be configured as proxyuser in HDFS. Different from the
'hdfs' user, the 'httpfs' user do not have blanket access to all HDFS files, only to the files
of users that can proxy-user as and with HDFS permissions being enforced. Also, the 'httpfs'
user does not have access to all encrypted files, which the 'hdfs' user does. The same holds
for Oozie, Templeton, HiveServer2, Knox and any other service that needs proxyuser config
in HDFS.

Regarding returning encrypted data back to the HTTP client. Well, that would mean that you
cannot simply use tools/libraries like curl/libcurl to integrate via the WebHDFS protocol
anymore. You'll need a client library that interacts with KMS to decrypt the encrypted key
and use libopenssl to decrypt. And if you are accessing file ranges, you'll have to know how
to manipulate the IV. IMO, going this path, completely defeats the motivation out of which
WebHDFS came to be.

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0, 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Charles Lamb
>         Attachments: HDFS-6134.001.patch, HDFS-6134.002.patch, HDFS-6134_test_plan.pdf,
HDFSDataatRestEncryption.pdf, HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message