hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jose Luis Soler <joso...@gmail.com>
Subject TraskTrackers Classpath
Date Mon, 13 Feb 2012 21:27:13 GMT
Hi All,

I'm quite new at using Hadoop and I'm facing a weird problem.

When running Hadoop in *local mode*, there is no problem getting a 
resource within the Mapper from *myJob.jar*, all works like a charm.

*this.getClass().getResource("/test1.txt").getFile()*

But when running in *pseudo-dist mode*, the above sentence is always 
unable to find the resource so there is no way for me to get the 
resource required in the Mapper.

As you can see below, the TaskTracker tries to get the resource from a 
weird place when running in pseudo-dist mode.

*Dir layout for the job attempt:*

josocas
     ??? distcache
     ??? jobcache
         ??? job_201202132214_0003
             ??? attempt_201202132214_0003_m_000000_1
             ?   ??? work
             ??? attempt_201202132214_0003_m_000001_1
             ?   ??? work
             ??? jars
             ?   ??? job.jar
             ??? jobToken
             ??? job.xml
             ??? work

*Stack trace:*

12/02/13 22:19:30 INFO mapred.JobClient: Task Id : 
attempt_201202132214_0003_m_000000_1, Status : FAILED
java.io.IOException: Resources not found 
!!!!*file:/tmp/hadoop-josocas/mapred/local/taskTracker/josocas/jobcache/job_201202132214_0003/jars/job.jar!/test1.txt*
     at es.josocas.hadoop.TokenizingMapper.map(TokenizingMapper.java:41)
     at es.josocas.hadoop.TokenizingMapper.map(TokenizingMapper.java:20)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
     at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:396)
     at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
     at org.apache.hadoop.mapred.Child.main(Child.java:264)


Thanks in advance for your help.

Regards,
Jose



Mime
View raw message