hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4417) add support for encrypted shuffle
Date Mon, 16 Jul 2012 22:28:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415721#comment-13415721
] 

Hadoop QA commented on MAPREDUCE-4417:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12536713/MAPREDUCE-4417.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified test files.

    -1 javac.  The applied patch generated 2115 javac compiler warnings (more than the trunk's
current 2066 warnings).

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    -1 findbugs.  The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    -1 core tests.  The patch failed these unit tests in hadoop-common-project/hadoop-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle:

                  org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
                  org.apache.hadoop.io.file.tfile.TestTFileByteArrays
                  org.apache.hadoop.mapred.TestFileInputFormatPathFilter
                  org.apache.hadoop.mapred.TestFieldSelection
                  org.apache.hadoop.mapred.TestBlockLimits
                  org.apache.hadoop.mapred.TestMiniMRClasspath
                  org.apache.hadoop.mapred.TestTextOutputFormat
                  org.apache.hadoop.mapred.TestSequenceFileInputFormat
                  org.apache.hadoop.mapreduce.TestMROutputFormat
                  org.apache.hadoop.mapreduce.lib.chain.TestChainErrors
                  org.apache.hadoop.mapreduce.lib.aggregate.TestMapReduceAggregates
                  org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
                  org.apache.hadoop.mapred.jobcontrol.TestJobControl
                  org.apache.hadoop.mapreduce.lib.input.TestMRSequenceFileAsTextInputFormat
                  org.apache.hadoop.mapred.TestMiniMRBringup
                  org.apache.hadoop.mapreduce.TestValueIterReset
                  org.apache.hadoop.mapreduce.lib.output.TestMRMultipleOutputs
                  org.apache.hadoop.mapred.TestMiniMRChildTask
                  org.apache.hadoop.mapred.TestMapRed
                  org.apache.hadoop.mapred.lib.TestMultipleOutputs
                  org.apache.hadoop.mapred.TestReporter
                  org.apache.hadoop.mapred.TestCollect
                  org.apache.hadoop.mapred.TestReduceFetch
                  org.apache.hadoop.mapred.TestNetworkedJob
                  org.apache.hadoop.mapred.TestTaskCommit
                  org.apache.hadoop.mapreduce.lib.output.TestFileOutputCommitter
                  org.apache.hadoop.mapred.TestClusterMRNotification
                  org.apache.hadoop.mapreduce.TestMapReduce
                  org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
                  org.apache.hadoop.mapred.TestJobCounters
                  org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat
                  org.apache.hadoop.mapred.TestMiniMRClientCluster
                  org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
                  org.apache.hadoop.mapreduce.lib.input.TestMultipleInputs
                  org.apache.hadoop.mapred.TestFileOutputCommitter
                  org.apache.hadoop.mapred.TestLazyOutput
                  org.apache.hadoop.mapred.TestLocalMRNotification
                  org.apache.hadoop.mapred.TestJobCleanup
                  org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
                  org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
                  org.apache.hadoop.mapred.lib.TestMultithreadedMapRunner
                  org.apache.hadoop.mapreduce.lib.chain.TestSingleElementChain
                  org.apache.hadoop.mapred.TestLineRecordReader
                  org.apache.hadoop.mapred.TestUserDefinedCounters
                  org.apache.hadoop.mapred.TestMapOutputType
                  org.apache.hadoop.mapred.lib.aggregate.TestAggregates
                  org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl
                  org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
                  org.apache.hadoop.mapreduce.lib.input.TestLineRecordReader
                  org.apache.hadoop.mapred.lib.TestChainMapReduce
                  org.apache.hadoop.mapred.TestClusterMapReduceTestCase
                  org.apache.hadoop.mapred.join.TestDatamerge
                  org.apache.hadoop.io.TestSequenceFileMergeProgress
                  org.apache.hadoop.mapreduce.lib.input.TestMRSequenceFileAsBinaryInputFormat
                  org.apache.hadoop.mapred.jobcontrol.TestLocalJobControl
                  org.apache.hadoop.mapreduce.lib.input.TestMRSequenceFileInputFilter
                  org.apache.hadoop.mapred.TestJavaSerialization
                  org.apache.hadoop.mapred.lib.TestKeyFieldBasedComparator
                  org.apache.hadoop.mapreduce.lib.join.TestJoinDatamerge
                  org.apache.hadoop.mapred.lib.TestLineInputFormat
                  org.apache.hadoop.mapreduce.lib.fieldsel.TestMRFieldSelection
                  org.apache.hadoop.mapred.TestJobSysDirWithDFS
                  org.apache.hadoop.mapred.TestComparators
                  org.apache.hadoop.mapreduce.lib.input.TestNLineInputFormat
                  org.apache.hadoop.mapred.TestMultipleTextOutputFormat
                  org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedComparator
                  org.apache.hadoop.mapreduce.lib.output.TestMRSequenceFileAsBinaryOutputFormat
                  org.apache.hadoop.mapreduce.lib.map.TestMultithreadedMapper
                  org.apache.hadoop.mapred.TestJobName
                  org.apache.hadoop.mapred.TestFileOutputFormat
                  org.apache.hadoop.mapreduce.security.TestJHSSecurity
                  org.apache.hadoop.mapred.TestMapProgress
                  org.apache.hadoop.mapreduce.lib.chain.TestMapReduceChain

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2603//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2603//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2603//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Javac warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2603//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2603//console

This message is automatically generated.
                
> add support for encrypted shuffle
> ---------------------------------
>
>                 Key: MAPREDUCE-4417
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4417
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2, security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>             Fix For: 2.1.0-alpha
>
>         Attachments: MAPREDUCE-4417.patch, MAPREDUCE-4417.patch
>
>
> Currently Shuffle fetches go on the clear. While Kerberos provides comprehensive authentication
for the cluster, it does not provide confidentiality. 
> When processing sensitive data confidentiality may be desired (at the expense of job
performance and resources utilization for doing encryption).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message