hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5759) IllegalArgumentException when CombineFileInputFormat is used as job InputFormat
Date Thu, 30 Apr 2009 08:09:30 GMT
IllegalArgumentException when CombineFileInputFormat is used as job InputFormat
-------------------------------------------------------------------------------

                 Key: HADOOP-5759
                 URL: https://issues.apache.org/jira/browse/HADOOP-5759
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
            Reporter: Amareshwari Sriramadasu
            Assignee: Amareshwari Sriramadasu
             Fix For: 0.21.0


As per my understanding, CombineFileInputFormat is creating splits with rackname as split
location. 
When I use CombineFileInputFormat as the InputFormat for job, job initialization fails with
following exception :
2009-04-28 14:10:40,162 ERROR mapred.EagerTaskInitializationListener (EagerTaskInitializationListener.java:run(83))
- Job initialization failed:
java.lang.IllegalArgumentException: Network location name contains /: /default-rack
  at org.apache.hadoop.net.NodeBase.set(NodeBase.java:76)
  at org.apache.hadoop.net.NodeBase.<init>(NodeBase.java:57)
  at org.apache.hadoop.mapred.JobTracker.addHostToNodeMapping(JobTracker.java:2342)
  at org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2336)
  at org.apache.hadoop.mapred.JobInProgress.createCache(JobInProgress.java:344)
  at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:441)
  at org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:81)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  at java.lang.Thread.run(Thread.java:619)

When I changed CombineFileInputFormat to pass just rackname (without '/'), JT wrongly resolves
 the node as /default-rack/<rack-name>.

Solution is to pass hostnames holding the block(on the rack),  instead of rackname.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message