hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mapred Learn <mapred.le...@gmail.com>
Subject Re: How to send files to task trackers in a map-red job
Date Fri, 10 Jun 2011 07:38:41 GMT
Found the solution:
Hadoop jar <> -conf <> -files <comma separated files on local/HDFS that go
in task trackers working dir>

Sent from my iPhone

On Jun 9, 2011, at 11:06 PM, Mapred Learn <mapred.learn@gmail.com> wrote:

> Resending, in case anybody has any inputs :::
> 
> 
> On Jun 9, 2011, at 5:35 PM, Mapred Learn <mapred.learn@gmail.com> wrote:
> 
>> Hi,
>> I have 2 files that I want to send to all tasktrackers during job execution.
>> I try something like:
>> hadoop jar abc.jar <main class> -conf <conf file> -cacheFile 'hdfs://<namenode>:port/user/jj/dummy/abc.dat#abc'
-cacheFile 'hdfs://<namenode>:port/user/jj/dummy/abc.txt#abc1'
>> 
>> But looks like I don't get second file to the task trackers and all task trackers
fail with exception - "file not found: /user/jj/dummy/abc.txt".
>> 
>> Could somebody please guide me to the right way how to get files to all the data
nodes in a map-red job ?
>> 
>> Thanks,
>> JJ

Mime
View raw message