hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Hadoop-streaming using binary executable c program
Date Fri, 22 Jul 2011 16:12:18 GMT
It looks like it tried to run your program and the program exited with a 1 not a 0.  What are
the stderr logs like for the mappers that were launched, you should be able to access them
through the Web GUI?  You might want to add in some stderr log messages to you c program too.
To be able to debug how far along it is going before exiting.

--Bobby Evans

On 7/22/11 9:19 AM, "Daniel Yehdego" <dtyehdego@miners.utep.edu> wrote:

I am trying to parallelize some very long RNA sequence for the sake of
predicting their RNA 2D structures. I am using a binary executable c
program called pknotsRG as my mapper. I tried the following bin/hadoop
command:

HADOOP_HOME$ bin/hadoop
jar /data/yehdego/hadoop-0.20.2/hadoop-0.20.2-streaming.jar
-mapper /data/yehdego/hadoop-0.20.2/pknotsRG
-file /data/yehdego/hadoop-0.20.2/pknotsRG
-input /user/yehdego/RF00028_B.bpseqL3G5_seg_Centered_Method.txt
-output /user/yehdego/RF-out -reducer NONE -verbose

but i keep getting the following error message:

java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
failed with code 1
        at
org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
        at
org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545)
        at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
        at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)

FYI: my input file is RF00028_B.bpseqL3G5_seg_Centered_Method.txt which
is a chunk of RNA sequences and the mapper is expected to get the input
and execute the input file line by line and out put the predicted
structure for each line of sequence for a specified number of maps. Any
help on this problem is really appreciated. Thanks.



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message