hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5952) LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex
Date Thu, 10 Jul 2014 09:07:04 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057288#comment-14057288
] 

Hadoop QA commented on MAPREDUCE-5952:
--------------------------------------

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12654958/MAPREDUCE-5952.v02.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4723//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4723//console

This message is automatically generated.

> LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for
mapOutIndex
> ------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5952
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5952
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am, mrv2
>    Affects Versions: 2.3.0
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>            Priority: Blocker
>         Attachments: MAPREDUCE-5952.v01.patch, MAPREDUCE-5952.v02.patch
>
>
> The javadoc comment for {{renameMapOutputForReduce}} incorrectly refers to a single map
output directory, whereas this depends on LOCAL_DIRS.
> mapOutIndex should be set to subMapOutputFile.getOutputIndexFile()
> {code}
> 2014-06-30 14:48:35,574 WARN [uber-SubtaskRunner] org.apache.hadoop.mapred.LocalContainerLauncher:
Exception running local (uberized) 'child' : java.io.FileNotFoundException: File /Users/gshegalov/workspace/hadoop-common/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobs/org.apache.hadoop.mapreduce.v2.
         TestMRJobs-localDir-nm-2_3/usercache/gshegalov/appcache/application_1404164272885_0001/output/file.out.index
does not exist
>   at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:517)
>   at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:726)
>   at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:507)
>   at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)                      
>   at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)                          
>   at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:334)    
>   at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:504)
>   at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.renameMapOutputForReduce(LocalContainerLauncher.java:471)
>   at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
>   at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:292)
>   at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:178)
>   at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:221)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)    
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)         
>   at java.util.concurrent.FutureTask.run(FutureTask.java:138)                       
>   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>   at java.lang.Thread.run(Thread.java:695)         
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message