hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5890) Support for encrypting Intermediate data and spills in local filesystem
Date Wed, 25 Jun 2014 18:34:25 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043884#comment-14043884
] 

Alejandro Abdelnur commented on MAPREDUCE-5890:
-----------------------------------------------

Fetcher.java
MapTask.java
MergerManagerImpl.java
Merger.java
ShuffleHandler.java
ShuffleHeader.java

* several space changes (configure your editor not to trim unmodified lines

CryptoUtils.java

* createIV(): javadocs, invalid params
* wrap() OUT/IN methods: any change to consolidate all/most signatures to delegate to a single
one doing the repetitive logic?
* a couple wrap() methods have a funny LOG message ####
* wrap() OUT methods use cc.AlgorithmBlockSize(), but wrap() IN methods use 16, for IN methods
you can use the cc already avail in the method.
* wrap() methods wrap if necessary (the IF ENCRYTPED has been moved inside), the name should
reflect that, maybe something like 'wrapIfNecessary()'

Fetcher.java

* copyMapOutput() is unconditionally correct the offset, this seems wrong.

* No need to define out2, just reuse out


> Support for encrypting Intermediate data and spills in local filesystem
> -----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5890
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5890
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 2.4.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Arun Suresh
>              Labels: encryption
>         Attachments: MAPREDUCE-5890.3.patch, MAPREDUCE-5890.4.patch, MAPREDUCE-5890.5.patch,
MAPREDUCE-5890.6.patch, MAPREDUCE-5890.7.patch, MAPREDUCE-5890.8.patch, org.apache.hadoop.mapred.TestMRIntermediateDataEncryption-output.txt,
syslog.tar.gz
>
>
> For some sensitive data, encryption while in flight (network) is not sufficient, it is
required that while at rest it should be encrypted. HADOOP-10150 & HDFS-6134 bring encryption
at rest for data in filesystem using Hadoop FileSystem API. MapReduce intermediate data and
spills should also be encrypted while at rest.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message