hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Larry McCay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Wed, 13 Aug 2014 17:06:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095719#comment-14095719

Larry McCay commented on HDFS-6134:

Thanks [~tucu00] that is pretty clear.

The question that remains for me is why this same scenario isn't achievable by the admin kinit'ing
as httpfs/HOST or Oozie or some other trusted proxy and then issuing a request with a doAs
user X.

We have to somehow fix this for webhdfs - it is an expected and valuable API and should remain
so with encrypted files without introducing a vulnerability.

Even if we have to do something like use another proxy (like Knox) and a shared secret to
ensure that there is additional verification of the origin of a KMS request from webhdfs.
This would enable proxies to access webhdfs resources with a signed/encrypted token - if KMS
gets a signed request from webhdfs that it can verify then it can proceed. The shared secret
can be made available through the credential provider API and webhdfs itself would just see
it as an opaque token that needs to be passed in the KMS request. Requiring an extra hop for
this access would be unfortunate too but if it is for additional security of the data it may
be acceptable.

Anyway, that's just a thought for keeping webhdfs as a first class citizen. We have to do

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0, 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Charles Lamb
>         Attachments: HDFS-6134.001.patch, HDFS-6134.002.patch, HDFS-6134_test_plan.pdf,
HDFSDataatRestEncryption.pdf, HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message