hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9331) Hadoop crypto codec framework and crypto codec implementations
Date Thu, 05 Sep 2013 02:06:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758681#comment-13758681

Andrew Purtell commented on HADOOP-9331:

Would it be possible for a Hadoop committer to comment on the viability of this issue and
related patches? 

There are HBASE-7544 and HIVE-4227/HIVE-5207 either depending on this framework or intent
to that effect stated on the respective issues.

In this framework, crypto codec implementations can be implemented and optimized in Hadoop
core instead of the JRE. This is a likely long term benefit because JRE crypto codecs must
be signed with a code signing certificate obtained under restrictive terms that must be controlled,
but Hadoop crypto codecs developed for this framework would not have this impediment.

Without a version of Hadoop containing this framework to target, upstream users may be forced
to seek alternative (and suboptimal, for the reason given above) implementation options. Or
we could see overlapping or competing frameworks that would lead in any case to wasted effort
and additional effort at rationalization. See https://issues.apache.org/jira/browse/HBASE-7544?focusedCommentId=13710611&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13710611
for an example.
> Hadoop crypto codec framework and crypto codec implementations
> --------------------------------------------------------------
>                 Key: HADOOP-9331
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9331
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0
>            Reporter: Jerry Chen
>         Attachments: Hadoop Crypto Design.pdf
>   Original Estimate: 504h
>  Remaining Estimate: 504h
> For use cases that deal with sensitive data, we often need to encrypt data to be stored
safely at rest. Hadoop common provides a codec framework for compression algorithms. We start
here. However because encryption algorithms require some additional configuration and methods
for key management, we introduce a crypto codec framework that builds on the compression codec
framework. It cleanly distinguishes crypto algorithms from compression algorithms, but shares
common interfaces between them where possible, and also carries extended interfaces where
necessary to satisfy those needs. We also introduce a generic Key type, and supporting utility
methods and classes, as a necessary abstraction for dealing with both Java crypto keys and
PGP keys.
> The task for this feature breaks into two parts:
> 1. The crypto codec framework that based on compression codec which can be shared by
all crypto codec implementations.
> 2. The codec implementations such as AES and others.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message