hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milind Bhandarkar <mili...@yahoo-inc.com>
Subject Re: Hadoop Streaming - running a jar file
Date Thu, 13 Nov 2008 00:10:43 GMT
You should specify A.jar on the bin/hadoop command line with "-file A.jar",
so that streaming knows to copy that file on the tasktracker node.

- milind


On 11/11/08 10:50 AM, "Amit_Gupta" <amitgupta151@gmail.com> wrote:

> 
> 
> Hi
> 
> I have a jar file which takes input from stdin and writes something on
> stdout. i.e. When I run
> 
> java -jar A.jar < input
> 
> It prints the required output.
> 
> However, when I run it as a mapper in hadoop streaming using the command
> 
> $HADOOP_HOME/bin/hadoop jar ....streaming.jar -input .. -output ...  -mapper
> 'java -jar A.jar'  -reducer NONE
> 
> i get the broken pipe exception.
> 
> 
> the error message is
> 
> additionalConfSpec_:null
> null=@@@userJobConfProps_.get(stream.shipped.hadoopstreaming
> packageJobJar:
> [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/dir/hadoop-hadoop/hadoop-unjar45410/]
> [] /tmp/streamjob45411.jar tmpDir=null
> 08/11/11 23:20:14 INFO mapred.FileInputFormat: Total input paths to process
> : 1
> 08/11/11 23:20:14 INFO streaming.StreamJob: getLocalDirs():
> [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/mapred]
> 08/11/11 23:20:14 INFO streaming.StreamJob: Running job:
> job_200811111724_0014
> 08/11/11 23:20:14 INFO streaming.StreamJob: To kill this job, run:
> 08/11/11 23:20:14 INFO streaming.StreamJob:
> /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job
> -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014
> 08/11/11 23:20:15 INFO streaming.StreamJob: Tracking URL:
> http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014
> 08/11/11 23:20:16 INFO streaming.StreamJob:  map 0%  reduce 0%
> 08/11/11 23:21:00 INFO streaming.StreamJob:  map 100%  reduce 100%
> 08/11/11 23:21:00 INFO streaming.StreamJob: To kill this job, run:
> 08/11/11 23:21:00 INFO streaming.StreamJob:
> /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job
> -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014
> 08/11/11 23:21:00 INFO streaming.StreamJob: Tracking URL:
> http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014
> 08/11/11 23:21:00 ERROR streaming.StreamJob: Job not Successful!
> 08/11/11 23:21:00 INFO streaming.StreamJob: killJob...
> Streaming Job Failed!
> 
> Could some one please help me with any ideas or pointers.
> 
> regards
> Amit
> 
> 
> --
> View this message in context:
> http://www.nabble.com/Hadoop-Streaming----running-a-jar-file-tp20445877p204458
> 77.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
> 


-- 
Milind Bhandarkar
Y!IM: GridSolutions
408-349-2136 
(milindb@yahoo-inc.com)


Mime
View raw message