hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2185) Infinite loop at creating splits using CombineFileInputFormat
Date Fri, 12 Nov 2010 20:13:14 GMT
Infinite loop at creating splits using CombineFileInputFormat
-------------------------------------------------------------

                 Key: MAPREDUCE-2185
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2185
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: job submission
            Reporter: Hairong Kuang
            Assignee: Hairong Kuang
             Fix For: 0.22.0


This is caused by a missing block in HDFS. So the block's locations are empty. The following
code adds the block to blockToNodes map but not to rackToBlocks map. Later on when generating
splits, only blocks in rackToBlocks are removed from blockToNodes map. So blockToNodes map
can never become empty therefore causing infinite loop

{code}
          // add this block to the block --> node locations map
          blockToNodes.put(oneblock, oneblock.hosts);

          // add this block to the rack --> block map
          for (int j = 0; j < oneblock.racks.length; j++) {
             ..
          }
{code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message