hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-11827) Encryption support for bulkloading data into table with encryption configured for hfile format 3
Date Wed, 27 Aug 2014 00:49:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111660#comment-14111660
] 

Andrew Purtell edited comment on HBASE-11827 at 8/27/14 12:49 AM:
------------------------------------------------------------------

The master key should not be available to any process or principal except the HBase service
daemons and account. Therefore I think this issue is invalid. This patch would require the
cluster master key be available to the potentially (i.e. probably) untrustworthy mapreduce
execution environment.

It's fine to bulk load unencrypted HFiles into an encrypted table. The regionservers determine
on a per file basis if something is encrypted or not. The bulk loaded files, even though they
are unencrypted in the beginning, can be read right alongside existing encrypted HFiles. To
have the regionserver encrypt the newly loaded HFiles, trigger a major compaction. Understood
that this requires a rewrite of the data that was just loaded in. It's necessary when only
the regionservers are trusted with sensitive key material.


was (Author: apurtell):
The master key should not be available to any process or principal except the HBase service
daemons and account. Therefore I think this issue is invalid. This patch would require the
cluster master key be available to the potentially (i.e. probably) untrustworthy mapreduce
execution environment.

It's fine to bulk load unencrypted HFiles into an encrypted table. The region servers determine
on a per file basis if something is encrypted or not. To have the region server encrypt the
bulk loaded data, trigger a major compaction.

> Encryption support for bulkloading data into table with encryption configured for hfile
format 3
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11827
>                 URL: https://issues.apache.org/jira/browse/HBASE-11827
>             Project: HBase
>          Issue Type: Improvement
>          Components: mapreduce
>    Affects Versions: 0.98.5
>            Reporter: Kashif J S
>            Assignee: Kashif J S
>             Fix For: 2.0.0, 0.98.7
>
>         Attachments: HBASE-11827-98-v1.patch, HBASE-11827-trunk-v1.patch
>
>
> The solution would be to add support to auto detect encryption parameters similar to
other parameters like compression, datablockencoding, etc when encryption is enabled for hfile
format 3. 
> The current patch does the following:
> 1. Automatically detects encryption type and key in HFileOutputFormat & HFileOutputFormat2.
> 2. Uses Base64encoder/decoder for url passing of Encryption key which is in bytes format



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message