hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hassen Riahi <hassen.ri...@cern.ch>
Subject PIPES
Date Tue, 24 Apr 2012 09:01:45 GMT
Hi all,

We have this error (*) when trying to execute the word-part example.  
Here is the content of  word-part.xml (**). We made also sure that / 
examples/bin/wordcount-part file exists in HDFS.
Any ideas?

Thanks for the help!
Hassen

(*)
$ /usr/lib/hadoop/bin/hadoop pipes -conf /usr/share/doc/hadoop-0.20/ 
examples/src/pipes/conf/word-part.xml -input /user/fasi/testROOT/pipes  
-output /user/fasi/testROOT/out/test1500
12/04/24 10:33:49 INFO security.UserGroupInformation: JAAS  
Configuration already set up for Hadoop, not re-installing.
12/04/24 10:33:49 WARN mapred.JobClient: No job jar file set.  User  
classes may not be found. See JobConf(Class) or JobConf#setJar(String).
12/04/24 10:33:50 WARN snappy.LoadSnappy: Snappy native library is  
available
12/04/24 10:33:50 INFO util.NativeCodeLoader: Loaded the native-hadoop  
library
12/04/24 10:33:50 INFO snappy.LoadSnappy: Snappy native library loaded
12/04/24 10:33:50 INFO mapred.FileInputFormat: Total input paths to  
process : 1
12/04/24 10:33:50 INFO mapred.JobClient: Running job:  
job_201204171750_0148
12/04/24 10:33:51 INFO mapred.JobClient:  map 0% reduce 0%
12/04/24 10:33:58 INFO mapred.JobClient: Task Id :  
attempt_201204171750_0148_m_000000_0, Status : FAILED
attempt_201204171750_0148_m_000000_0: Hadoop Pipes Exception: Unknown  
binary command 10 at /export/crawlspace/chris/work/branch-0.20/src/c++/ 
pipes/impl/HadoopPipes.cc:446 in virtual void  
HadoopPipes::BinaryProtocol::nextEvent()
12/04/24 10:33:58 INFO mapred.JobClient: Task Id :  
attempt_201204171750_0148_m_000001_0, Status : FAILED
java.io.IOException
	at  
org 
.apache 
.hadoop 
.mapred.pipes.OutputHandler.waitForAuthentication(OutputHandler.java: 
188)
	at  
org 
.apache 
.hadoop 
.mapred.pipes.Application.waitForAuthentication(Application.java:194)
	at org.apache.hadoop.mapred.pipes.Application.<init>(Application.java: 
149)
	at  
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java: 
68)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:416)
	at  
org 
.apache 
.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java: 
1157)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)

(**)
$ cat  /usr/share/doc/hadoop-0.20/examples/src/pipes/conf/word-part.xml
<?xml version="1.0"?>
<configuration>

<property>
   <name>mapred.reduce.tasks</name>
   <value>2</value>
</property>

<property>
   <name>hadoop.pipes.executable</name>
   <value>hdfs:/examples/bin/wordcount-part</value>
</property>

<property>
   <name>hadoop.pipes.java.recordreader</name>
   <value>true</value>
</property>

<property>
   <name>hadoop.pipes.java.recordwriter</name>
   <value>true</value>
</property>

</configuration>





Mime
View raw message