Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <SNT113-W38672B8C44F44D108649B5F94E0@phx.gbl>
Content-Type: multipart/alternative;
	boundary="_458e7e99-f49a-4c8f-a7a0-9445299bbf38_"
From: Daniel Yehdego <dtyehdego@miners.utep.edu>
To: <common-user@hadoop.apache.org>
Subject: RE: Hadoop-streaming using binary executable c program
Date: Fri, 22 Jul 2011 17:31:42 +0000
Importance: Normal
In-Reply-To: <CA4F0B92.26A1A%evans@yahoo-inc.com>
References: 
 <1311344389.20953.12.camel@bulgaria.gcl.cis.udel.edu>,<CA4F0B92.26A1A%evans@yahoo-inc.com>
MIME-Version: 1.0

--_458e7e99-f49a-4c8f-a7a0-9445299bbf38_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable


Hi Bobby=2C Thanks for the response.

After I tried the following comannd:

bin/hadoop jar $HADOOP_HOME/hadoop-0.20.2-streaming.jar -mapper /data/yehde=
go/hadoop-0.20.2/pknotsRG-1.3/src/pknotsRG -  -file /data/yehdego/hadoop-0.=
20.2/pknotsRG-1.3/src/pknotsRG  -reducer NONE -input /user/yehdego/RNAData/=
RF00028_B.bpseqL3G5_seg_Centered_Method.txt -output /user/yehdego/RF-out - =
verbose

I got a stderr logs :

java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess fail=
ed with code 139
	at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.jav=
a:311)
	at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:5=
45)
	at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
	at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)


syslog logs

2011-07-22 13:02:27=2C467 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: In=
itializing JVM Metrics with processName=3DMAP=2C sessionId=3D
2011-07-22 13:02:27=2C913 INFO org.apache.hadoop.mapred.MapTask: numReduceT=
asks: 0
2011-07-22 13:02:28=2C149 INFO org.apache.hadoop.streaming.PipeMapRed: Pipe=
MapRed exec [/data/yehdego/hadoop_tmp/dfs/local/taskTracker/jobcache/job_20=
1107181535_0079/attempt_201107181535_0079_m_000000_0/work/./pknotsRG]
2011-07-22 13:02:28=2C242 INFO org.apache.hadoop.streaming.PipeMapRed: R/W/=
S=3D1/0/0 in:NA [rec/s] out:NA [rec/s]
2011-07-22 13:02:28=2C267 INFO org.apache.hadoop.streaming.PipeMapRed: MROu=
tputThread done
2011-07-22 13:02:28=2C267 INFO org.apache.hadoop.streaming.PipeMapRed: MREr=
rorThread done
2011-07-22 13:02:28=2C267 INFO org.apache.hadoop.streaming.PipeMapRed: Pipe=
MapRed failed!
2011-07-22 13:02:28=2C361 WARN org.apache.hadoop.mapred.TaskTracker: Error =
running child
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess fail=
ed with code 139
	at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.jav=
a:311)
	at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:5=
45)
	at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
	at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)
2011-07-22 13:02:28=2C395 INFO org.apache.hadoop.mapred.TaskRunner: Runnnin=
g cleanup for the task


Regards=2C=20

Daniel T. Yehdego
Computational Science Program=20
University of Texas at El Paso=2C UTEP=20
dtyehdego@miners.utep.edu

> From: evans@yahoo-inc.com
> To: common-user@hadoop.apache.org=3B dtyehdego@miners.utep.edu
> Date: Fri=2C 22 Jul 2011 09:12:18 -0700
> Subject: Re: Hadoop-streaming using binary executable c program
>=20
> It looks like it tried to run your program and the program exited with a =
1 not a 0.  What are the stderr logs like for the mappers that were launche=
d=2C you should be able to access them through the Web GUI?  You might want=
 to add in some stderr log messages to you c program too. To be able to deb=
ug how far along it is going before exiting.
>=20
> --Bobby Evans
>=20
> On 7/22/11 9:19 AM=2C "Daniel Yehdego" <dtyehdego@miners.utep.edu> wrote:
>=20
> I am trying to parallelize some very long RNA sequence for the sake of
> predicting their RNA 2D structures. I am using a binary executable c
> program called pknotsRG as my mapper. I tried the following bin/hadoop
> command:
>=20
> HADOOP_HOME$ bin/hadoop
> jar /data/yehdego/hadoop-0.20.2/hadoop-0.20.2-streaming.jar
> -mapper /data/yehdego/hadoop-0.20.2/pknotsRG
> -file /data/yehdego/hadoop-0.20.2/pknotsRG
> -input /user/yehdego/RF00028_B.bpseqL3G5_seg_Centered_Method.txt
> -output /user/yehdego/RF-out -reducer NONE -verbose
>=20
> but i keep getting the following error message:
>=20
> java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
> failed with code 1
>         at
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:=
311)
>         at
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545=
)
>         at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:1=
32)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
>         at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.ja=
va:36)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358=
)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
>=20
> FYI: my input file is RF00028_B.bpseqL3G5_seg_Centered_Method.txt which
> is a chunk of RNA sequences and the mapper is expected to get the input
> and execute the input file line by line and out put the predicted
> structure for each line of sequence for a specified number of maps. Any
> help on this problem is really appreciated. Thanks.
>=20
>=20
 		 	   		  =

--_458e7e99-f49a-4c8f-a7a0-9445299bbf38_--