Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Wed, 25 Jun 2014 18:34:25 +0000 (UTC)
From: "Alejandro Abdelnur (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12714395.1400126729268.48521.1403721265388@arcas>
In-Reply-To: <JIRA.12714395.1400126729268@arcas>
References: <JIRA.12714395.1400126729268@arcas>
Subject: [jira] [Commented] (MAPREDUCE-5890) Support for encrypting
 Intermediate data and spills in local filesystem
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043884#comment-14043884 ] 

Alejandro Abdelnur commented on MAPREDUCE-5890:
-----------------------------------------------

Fetcher.java
MapTask.java
MergerManagerImpl.java
Merger.java
ShuffleHandler.java
ShuffleHeader.java

* several space changes (configure your editor not to trim unmodified lines

CryptoUtils.java

* createIV(): javadocs, invalid params
* wrap() OUT/IN methods: any change to consolidate all/most signatures to delegate to a single one doing the repetitive logic?
* a couple wrap() methods have a funny LOG message ####
* wrap() OUT methods use cc.AlgorithmBlockSize(), but wrap() IN methods use 16, for IN methods you can use the cc already avail in the method.
* wrap() methods wrap if necessary (the IF ENCRYTPED has been moved inside), the name should reflect that, maybe something like 'wrapIfNecessary()'

Fetcher.java

* copyMapOutput() is unconditionally correct the offset, this seems wrong.

* No need to define out2, just reuse out


> Support for encrypting Intermediate data and spills in local filesystem
> -----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5890
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5890
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 2.4.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Arun Suresh
>              Labels: encryption
>         Attachments: MAPREDUCE-5890.3.patch, MAPREDUCE-5890.4.patch, MAPREDUCE-5890.5.patch, MAPREDUCE-5890.6.patch, MAPREDUCE-5890.7.patch, MAPREDUCE-5890.8.patch, org.apache.hadoop.mapred.TestMRIntermediateDataEncryption-output.txt, syslog.tar.gz
>
>
> For some sensitive data, encryption while in flight (network) is not sufficient, it is required that while at rest it should be encrypted. HADOOP-10150 & HDFS-6134 bring encryption at rest for data in filesystem using Hadoop FileSystem API. MapReduce intermediate data and spills should also be encrypted while at rest.


--
This message was sent by Atlassian JIRA
(v6.2#6252)