hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Encryption in HDFS
Date Tue, 26 Feb 2013 05:24:43 GMT
The following JIRAs are related to your research:

HADOOP-9331: Hadoop crypto codec framework and crypto codec implementations<
https://issues.apache.org/jira/browse/hadoop-9331> and related sub-tasks

MAPREDUCE-5025: Key Distribution and Management for supporting crypto codec
in Map Reduce<https://issues.apache.org/jira/browse/mapreduce-5025> and
related JIRAs

On Mon, Feb 25, 2013 at 9:10 PM, Seonyeong Bak <renderaid@gmail.com> wrote:

> Hello, I'm a university student.
> I implemented AES and Triple DES with CompressionCodec in java
> cryptography architecture (JCA)
> The encryption is performed by a client node using Hadoop API.
> Map tasks read blocks from HDFS and these blocks are decrypted by each map
> tasks.
> I tested my implementation with generic HDFS.
> My cluster consists of 3 nodes (1 master node, 3 worker nodes) and each
> machines have quad core processor (i7-2600) and 4GB memory.
> A test input is 1TB text file which consists of 32 multiple text files (1
> text file is 32GB)
> I expected that the encryption takes much more time than generic HDFS.
> The performance does not differ significantly.
> The decryption step takes about 5-7% more than generic HDFS.
> The encryption step takes about 20-30% more than generic HDFS because it
> is implemented by single thread and executed by 1 client node.
> So the encryption can get more performance.
> May there be any error in my test?
> I know there are several implementation for encryting files in HDFS.
> Are these implementations enough to secure HDFS?
> best regards,
> seonpark
> * Sorry for my bad english

View raw message