hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayra Mendoza <aryam...@gmail.com>
Subject Hadoop pipes and custom imput format
Date Wed, 03 Feb 2010 20:43:29 GMT
Hi,

I'm trying to run an c++ program with pipe (open and save images). I'm using
the WholeFileInputFormat, this is my execution line:

*Hadoop pipes -conf EscGris.xml -libjars pipe.jar -input input -output
output*

pipe.jar has WholeFileInputFormat, WholeFileRecordReader and
NullOutputFormat classes

*My configuration file is:*
<?xml version="1.0"?>
<configuration>
  <property>
  <name>mapred.reduce.tasks</name>
  <value>0</value>
</property>
<property>
    <name>hadoop.pipes.executable</name>
    <value>EscGris</value>
</property>
<property>
    <name>mapred.input.format.class</name>
    <value>pipe.WholeFileInputFormat</value>
</property>
<property>
    <name>mapred.output.format.class</name>
    <value>pipe.NullOutputFormat</value>
</property>
<property>
  <name>hadoop.pipes.java.recordreader</name>
  <value>false</value>
</property>
<property>
  <name>hadoop.pipes.java.mapper</name>
  <value>false</value>
</property>
<property>
  <name>hadoop.pipes.java.reducer</name>
  <value>false</value>
</property>
<property>
  <name>keep.failed.task.files</name>
  <value>true</value>
</property>
<property>
  <name>mapred.system.dir</name>
  <value>/tmp/hadoop/mapred/system</value>
</property>
<property>
  <name>tmpjars</name>
  <value>home/training/ProyectoGrado/03_ImgEscGris/c++/pipe.jar</value>
</property>
</configuration>

*But I get the following error:*

attempt_201002030521_0016_m_000000_0:* Hadoop Pipes Exception: RecordReader
not defined* at
/home/oom/work/eclipse/hadoop-20/src/c++/pipes/impl/HadoopPipes.cc:691 in
virtual void HadoopPipes::TaskContextImpl::runMap(std::string, int, bool)
10/02/03 12:01:07 INFO mapred.JobClient: Task Id :
attempt_201002030521_0016_m_000001_0, Status : FAILED
java.io.IOException: pipe child exception
    at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:151)
    at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:101)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readByte(DataInputStream.java:250)
    at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:298)
    at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:319)
    at
org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:114)


*What am I doing wrong???*

Thank's a lot =0)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message