hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7544) Transparent table/CF encryption
Date Tue, 15 Apr 2014 19:06:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13969917#comment-13969917

Andrew Purtell commented on HBASE-7544:

For an upcoming talk on security features I went back and looked at the impact of WAL encryption
on more recent JVMs and after the changes to the WAL threading model that went in to 0.98+.
I had to resort to a dual core mobile CPU with hyperthreading from ~2010 (with cpufreq locked
at max) at the moment since Amazon HVMs don't give access to perf hw registers, but I plan
to retest on bare Haswell server hardware. 

Three runs averaged, HLogPerformanceEvaluation -keySize 50 -valueSize 100 -threads 100 -iterations
1000000 ( -encryption AES )
VM flags: -XX:+UseG1GC -XX:+UseAES -XX:+UseAESIntrinsics (AES flags where supported)

||Test||Throughput ops/sec||Total cycles||Insns per cycle||
|Oracle Java 1.7.0_45-b18 - None|52658.302|8878179986750|0.47|
|Oracle Java 1.7.0_45-b18 - AES WAL encryption|48045.834|9911748458387|0.57|
|OpenJDK 1.8.0_20-b09 - None|54874.125|8662634367005|0.46|
|OpenJDK 1.8.0_20-b09 - AES WAL encryption|50659.507|9668111259270|0.61|

What is interesting are the relative differences in later test cases from the first test case.
Though there is more work per edit to do with encryption enabled by definition, for this microbenchmark
the throughput of 8u20 with WAL encryption and AES intrinsics enabled is only ~4% off from
7u45 with no WAL encryption because of native code generation improvements on AES-NI capable
hardware. Ops/sec measurements vary ~1.5% from run to run.

> Transparent table/CF encryption
> -------------------------------
>                 Key: HBASE-7544
>                 URL: https://issues.apache.org/jira/browse/HBASE-7544
>             Project: HBase
>          Issue Type: New Feature
>          Components: HFile, io
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 0.98.0
>         Attachments: 7544-addendum-1.patch, 7544-final.patch, 7544.patch, 7544.patch,
7544.patch, 7544.patch, 7544.patch, 7544.patch, 7544p1.patch, 7544p1.patch, 7544p2.patch,
7544p2.patch, 7544p3.patch, 7544p3.patch, 7544p4.patch, historical-7544.patch, historical-7544.pdf,
historical-shell.patch, latency-single.7544.xlsx
> Introduce transparent encryption of HBase on disk data.
> Depends on a separate contribution of an encryption codec framework to Hadoop core and
an AES-NI (native code) codec. This is work done in the context of MAPREDUCE-4491 but I'd
gather there will be additional JIRAs for common and HDFS parts of it.
> Requirements:
> - Transparent encryption at the CF or table level
> - Protect against all data leakage from files at rest
> - Two-tier key architecture for consistency with best practices for this feature in the
RDBMS world
> - Built-in key management
> - Flexible and non-intrusive key rotation
> - Mechanisms not exposed to or modifiable by users
> - Hardware security module integration (via Java KeyStore)
> - HBCK support for transparently encrypted files (+ plugin architecture for HBCK)
> Additional goals:
> - Shell support for administrative functions
> - Avoid performance impact for the null crypto codec case
> - Play nicely with other changes underway: in HFile, block coding, etc.
> We're aiming for rough parity with Oracle's transparent tablespace encryption feature,
described in http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
> {quote}
> “Transparent Data Encryption uses a 2-tier key architecture for flexible and non-intrusive
key rotation and least operational and performance impact: Each application table with at
least one encrypted column has its own table key, which is applied to all encrypted columns
in that table. Equally, each encrypted tablespace has its own tablespace key. Table keys are
stored in the data dictionary of the database, while tablespace keys are stored in the header
of the tablespace and additionally, the header of each underlying OS file that makes up the
tablespace.  Each of these keys is encrypted with the TDE master encryption key, which is
stored outside of the database in an external security module: either the Oracle Wallet (a
PKCS#12 formatted file that is encrypted using a passphrase supplied either by the designated
security administrator or DBA during setup),  or a Hardware Security Module (HSM) device for
higher assurance […]”
> {quote}
> Further design details forthcoming in a design document and patch as soon as we have
all of the clearances in place.

This message was sent by Atlassian JIRA

View raw message