hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2185) Infinite loop at creating splits using CombineFileInputFormat
Date Mon, 23 May 2011 22:57:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038301#comment-13038301
] 

Hadoop QA commented on MAPREDUCE-2185:
--------------------------------------

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12480153/MAPREDUCE-2185.patch
  against trunk revision 1126591.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

    +1 system test framework.  The patch passed system test framework compile.

Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/297//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/297//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/297//console

This message is automatically generated.

> Infinite loop at creating splits using CombineFileInputFormat
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-2185
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2185
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: job submission
>            Reporter: Hairong Kuang
>            Assignee: Ramkumar Vadali
>         Attachments: MAPREDUCE-2185.patch
>
>
> This is caused by a missing block in HDFS. So the block's locations are empty. The following
code adds the block to blockToNodes map but not to rackToBlocks map. Later on when generating
splits, only blocks in rackToBlocks are removed from blockToNodes map. So blockToNodes map
can never become empty therefore causing infinite loop
> {code}
>           // add this block to the block --> node locations map
>           blockToNodes.put(oneblock, oneblock.hosts);
>           // add this block to the rack --> block map
>           for (int j = 0; j < oneblock.racks.length; j++) {
>              ..
>           }
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message