hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6134) Transparent data at rest encryption
Date Thu, 26 Jun 2014 16:04:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044797#comment-14044797

Owen O'Malley commented on HDFS-6134:

Mike, I remember you from when I interviewed you.

You are talking about collisions between IV's, not key space. By using either 32 bytes of
randomness (if someone is worried about crypto attacks there is no excuse not to use AES256),
there is *NO* possibility of collision even assuming an insanely bad practice of using a single
key version for a huge number of files. I obviously understand and applied the birthday paradox
to get the numbers.

Note that we *already* have key rolling and the key is already a random string of bytes. Adding
additional layers of randomness just gives the appearance of more security. That may be wonderful
in the closed source security world, but it actively harmful in open source. In open source
having a clear implementation that is open for inspection is by far the best protection. 

Note that the other issue with not using the keys as intended is that many Hadoop users launch
jobs that read millions of files. We can't afford to have the client fetch a different key
for each of those millions of files.

> Transparent data at rest encryption
> -----------------------------------
>                 Key: HDFS-6134
>                 URL: https://issues.apache.org/jira/browse/HDFS-6134
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 2.3.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>         Attachments: HDFSDataatRestEncryptionProposal_obsolete.pdf, HDFSEncryptionConceptualDesignProposal-2014-06-20.pdf
> Because of privacy and security regulations, for many industries, sensitive data at rest
must be in encrypted form. For example: the health­care industry (HIPAA regulations), the
card payment industry (PCI DSS regulations) or the US government (FISMA regulations).
> This JIRA aims to provide a mechanism to encrypt HDFS data at rest that can be used transparently
by any application accessing HDFS via Hadoop Filesystem Java API, Hadoop libhdfs C library,
> The resulting implementation should be able to be used in compliance with different regulation

This message was sent by Atlassian JIRA

View raw message