hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-7544) Transparent table/CF encryption
Date Mon, 04 Mar 2013 09:37:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592085#comment-13592085
] 

Andrew Purtell edited comment on HBASE-7544 at 3/4/13 9:36 AM:
---------------------------------------------------------------

Feedback from the Feb 28 HUG: Row key data may leak into encoded region names in META and
in ZooKeeper znodes. We have not addressed this yet, mainly because of the challenge of dealing
with META. It should be straightforward to encrypt znode data on write and decrypt on read.
For META we cannot change the region name encoding without disrupting sort order. The solution
for obscuring on disk META data is for the admin to enable encryption on the META table (and
for HBase to support META schema configuration changes).

We may simply want to clearly document that constructing row keys with sensitive data should
be avoided, as it may leak among users of the system. 

Transparent encryption does not address protection of the data of one user from another. This
is outside the scope of this JIRA. To address this other use case, we might propose HTable
support for compression codecs for mutation data. Aside from being useful for transparent
compression, encryption codecs can stand in for compression codecs, thus the user can at their
option encrypt keys and data. (It's an application concern so HTable support for this would
be convenient but not essential.) Encrypting keys will have obvious consequences that should
still be documented.
                
      was (Author: apurtell):
    Feedback from the Feb 28 HUG: Row key data may leak into encoded region names in META
and in ZooKeeper znodes. We have not addressed this yet, mainly because of the challenge of
dealing with META. It should be straightforward to encrypt znode data on write and decrypt
on read. For META we cannot change the region name encoding without disrupting sort order.
The solution for obscuring on disk META data is for the admin to enable encryption on the
META table (and for HBase to support META schema configuration changes).

We may simply want to clearly document that constructing row keys with sensitive data should
be avoided, as it may leak among users of the system. 

Transparent encryption does not address protection of the data of one user from another. For
this we might propose HTable support for compression codecs for mutation data. Aside from
being useful for transparent compression, encryption codecs can stand in for compression codecs,
thus the user can at their option encrypt keys and data. (It's an application concern so HTable
support for this would be convenient but not essential.) Encrypting keys will have obvious
consequences that should still be documented.
                  
> Transparent table/CF encryption
> -------------------------------
>
>                 Key: HBASE-7544
>                 URL: https://issues.apache.org/jira/browse/HBASE-7544
>             Project: HBase
>          Issue Type: New Feature
>          Components: HFile, io
>    Affects Versions: 0.96.0
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>         Attachments: 7544.patch, 7544.pdf
>
>
> Introduce transparent encryption of HBase on disk data.
> Depends on a separate contribution of an encryption codec framework to Hadoop core and
an AES-NI (native code) codec. This is work done in the context of MAPREDUCE-4491 but I'd
gather there will be additional JIRAs for common and HDFS parts of it.
> Requirements:
> - Transparent encryption at the CF or table level
> - Protect against all data leakage from files at rest
> - Two-tier key architecture for consistency with best practices for this feature in the
RDBMS world
> - Built-in key management
> - Flexible and non-intrusive key rotation
> - Mechanisms not exposed to or modifiable by users
> - Hardware security module integration (via Java KeyStore)
> - HBCK support for transparently encrypted files (+ plugin architecture for HBCK)
> Additional goals:
> - Shell support for administrative functions
> - Avoid performance impact for the null crypto codec case
> - Play nicely with other changes underway: in HFile, block coding, etc.
> We're aiming for rough parity with Oracle's transparent tablespace encryption feature,
described in http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
as
> {quote}
> “Transparent Data Encryption uses a 2-tier key architecture for flexible and non-intrusive
key rotation and least operational and performance impact: Each application table with at
least one encrypted column has its own table key, which is applied to all encrypted columns
in that table. Equally, each encrypted tablespace has its own tablespace key. Table keys are
stored in the data dictionary of the database, while tablespace keys are stored in the header
of the tablespace and additionally, the header of each underlying OS file that makes up the
tablespace.  Each of these keys is encrypted with the TDE master encryption key, which is
stored outside of the database in an external security module: either the Oracle Wallet (a
PKCS#12 formatted file that is encrypted using a passphrase supplied either by the designated
security administrator or DBA during setup),  or a Hardware Security Module (HSM) device for
higher assurance […]”
> {quote}
> Further design details forthcoming in a design document and patch as soon as we have
all of the clearances in place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message