hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From venkata subbarayudu <avsrit2...@gmail.com>
Subject Hadoop streaming command : -file option to pass a directory to jobcache
Date Thu, 18 Mar 2010 05:53:56 GMT
Hi All,
       I am new to hadoop and is using Python to write MapReduce tasks. In
order to execute the streaming command I am using the following command.

bin/hadoop jar hadoop-0.20.0-streaming.jar -mapper pkg2Cls.py -jobconf
mapred.map.tasks=5 -jobconf mapred.reduce.tasks=0 -input
/usr/test/linecount  -output linecountresults -file pkg2Cls.py -file
pkg1Cls.py

which is working fine. But now I want to pass the the entire directory of my
python files to the "-file option", instead of passing each file using the
-file option.

how can I do this.


Thanks for your help in advance.
Subbarayudu Amanchi.

Mime
View raw message