hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: Error while running Hadoop Source Code
Date Thu, 07 Nov 2013 00:58:53 GMT
Don't see anything in the logs that you pasted.

Can you paste the following in say pastebin?
 - All of the TaskTracker log
 - The task-logs. These are syslog, stderr, stdout files for a specific TaskAttempt.
 - And specific TaskAttempt's TaskAttemtpID that is failing.

Thanks,
+Vinod

On Nov 6, 2013, at 2:50 PM, Basu,Indrashish wrote:

>  
> Can anyone please assist regarding this ?
> 
> Thanks in advance
> 
> Regards,
> 
> Indra
> 
>  
>  
> On Wed, 06 Nov 2013 09:50:02 -0500, Basu,Indrashish wrote:
> 
>>  
>> Hi Vinod,
>> 
>> Thanks for your help regarding this. I checked the task logs, this is what it is
giving as output.
>> 
>>  
>> 2013-11-06 06:40:05,541 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed
JVM ID: jvm_201311060636_0001_m_1100862588
>> 2013-11-06 06:40:05,553 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_1100862588
spawned.
>> 2013-11-06 06:40:05,650 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed
JVM ID: jvm_201311060636_0001_m_-1960039766
>> 2013-11-06 06:40:05,651 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_-1960039766
spawned.
>> 2013-11-06 06:40:07,496 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_1100862588
given task: attempt_201311060
>> 636_0001_m_000001_3
>> 2013-11-06 06:40:07,618 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_-1960039766
given task: attempt_20131106
>> 0636_0001_m_000000_3
>> 2013-11-06 06:40:08,013 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false
>> 2013-11-06 06:40:08,014 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from task: false
>> 2013-11-06 06:40:08,015 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false
>> 2013-11-06 06:40:08,015 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from task: false
>> 2013-11-06 06:40:09,361 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311060636_0001_m_000001_3
0.0%
>> 2013-11-06 06:40:09,735 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201311060636_0001_m_000000_3
0.0%
>> 2013-11-06 06:40:10,018 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201311060636_0001_m_1100862588
exited. Number of tasks it ran: 0
>> 2013-11-06 06:40:11,021 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201311060636_0001_m_-1960039766
exited. Number of tasks it ran: 0
>> 2013-11-06 06:40:11,442 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false
>> 2013-11-06 06:40:11,442 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from task: false
>> 2013-11-06 06:40:11,443 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false
>> 2013-11-06 06:40:11,443 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from task: false
>> 2013-11-06 06:40:13,021 INFO org.apache.hadoop.mapred.TaskRunner: attempt_201311060636_0001_m_000001_3
done; removing files.
>> 2013-11-06 06:40:13,037 INFO org.apache.hadoop.mapred.TaskTracker: addCPUFreeSlot
: current free slots : 3
>> 2013-11-06 06:40:14,025 INFO org.apache.hadoop.mapred.TaskRunner: attempt_201311060636_0001_m_000000_3
done; removing files.
>> 2013-11-06 06:40:14,028 INFO org.apache.hadoop.mapred.TaskTracker: addCPUFreeSlot
: current free slots : 4
>> 2013-11-06 06:40:14,476 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false
>> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from task: false
>> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false
>> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from task: false
>> 2013-11-06 06:40:14,894 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
: false
>> 2013-11-06 06:40:14,900 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction
(registerTask): attempt_201311060636_0001_m_000002_0 task's
>> state:UNASSIGNED
>> 2013-11-06 06:40:14,902 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch
: attempt_201311060636_0001_m_000002_0 on CPU
>> 2013-11-06 06:40:14,904 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher,
current free CPU slots : 4 and trying to launch attempt_2013
>> 11060636_0001_m_000002_0
>> 2013-11-06 06:40:14,909 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction
for task: attempt_201311060636_0001_m_000000_3
>> 2013-11-06 06:40:14,920 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction
for task: attempt_201311060636_0001_m_000001_3
>> 2013-11-06 06:40:15,161 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed
JVM ID: jvm_201311060636_0001_m_164532908
>> 2013-11-06 06:40:15,162 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201311060636_0001_m_164532908
spawned.
>> 2013-11-06 06:40:17,216 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201311060636_0001_m_164532908
given task: attempt_2013110606
>> 36_0001_m_000002_0
>>  
>> Regards,
>> 
>> Indrashish
>> 
>>  
>> On Tue, 5 Nov 2013 10:09:36 -0800, Vinod Kumar Vavilapalli wrote:
>> 
>> It seems like your pipes mapper is exiting before consuming all the input. Did you
check the task-logs on the web UI?
>> 
>> Thanks,
>> +Vinod
>> 
>> On Nov 5, 2013, at 7:25 AM, Basu,Indrashish wrote:
>> 
>> 
>> Hi,
>> 
>> Can anyone kindly assist on this ?
>> 
>> Regards,
>> Indrashish
>> 
>> 
>> On Mon, 04 Nov 2013 10:23:23 -0500, Basu,Indrashish wrote:
>> Hi All,
>> Any update on the below post ?
>> I came across some old post regarding the same issue. It explains the
>> solution as " The *nopipe* example needs more documentation.  It
>> assumes that it is run with the InputFormat from
>> src/test/org/apache/*hadoop*/mapred/*pipes*/
>> *WordCountInputFormat*.java, which has a very specific input split
>> format. By running with a TextInputFormat, it will send binary bytes
>> as the input split and won't work right. The *nopipe* example should
>> probably be recoded *to* use libhdfs *too*, but that is more
>> complicated *to* get running as a unit test. Also note that since the
>> C++ example is using local file reads, it will only work on a cluster
>> if you have nfs or something working across the cluster. "
>> I would need some more light on the above explanation , so if anyone
>> could elaborate a bit about the same as what needs to be done exactly.
>> To mention, I am trying to run a sample KMeans algorithm on a GPU
>> using Hadoop.
>> Thanks in advance.
>> Regards,
>> Indrashish.
>> On Thu, 31 Oct 2013 20:00:10 -0400, Basu,Indrashish wrote:
>> Hi,
>> I am trying to run a sample Hadoop GPU source code (kmeans algorithm)
>> on an ARM processor and getting the below error. Can anyone please
>> throw some light on this ?
>> rmr: cannot remove output: No such file or directory.
>> 13/10/31 13:43:12 WARN mapred.JobClient: No job jar file set.  User
>> classes may not be found. See JobConf(Class) or
>> JobConf#setJar(String).
>> 13/10/31 13:43:12 INFO mapred.FileInputFormat: Total input paths to
>> process : 1
>> 13/10/31 13:43:13 INFO mapred.JobClient: Running job: job_201310311320_0001
>> 13/10/31 13:43:14 INFO mapred.JobClient:  map 0% reduce 0%
>> 13/10/31 13:43:39 INFO mapred.JobClient: Task Id :
>> attempt_201310311320_0001_m_000000_0, Status : FAILED
>> java.io.IOException: pipe child exception
>>    at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:191)
>>    at
>> org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:103)
>>    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363)
>>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>>    at org.apache.hadoop.mapred.Child.main(Child.java:170)
>> Caused by: java.net.SocketException: Broken pipe
>>    at java.net.SocketOutputStream.socketWrite0(Native Method)
>>    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
>>    at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
>>    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>    at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
>>    at java.io.DataOutputStream.write(DataOutputStream.java:107)
>>    at
>> org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333)
>>    at
>> org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286)
>>    at
>> org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92)
>>    ... 3 more
>> attempt_201310311320_0001_m_000000_0: cmd: [bash, -c, exec
>> '/app/hadoop/tmp/mapred/local/taskTracker/archive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans2D'
>> '0'  < /dev/null  1>>
>> /usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/attempt_201310311320_0001_m_000000_0/stdout
>> 2>> /usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/
>> Regards,
>> 
>> -- 
>> Indrashish Basu
>> Graduate Student
>> Department of Electrical and Computer Engineering
>> University of Florida
>> 
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to which
it is addressed and may contain information that is confidential, privileged and exempt from
disclosure under applicable law. If the reader of this message is not the intended recipient,
you are hereby notified that any printing, copying, dissemination, distribution, disclosure
or forwarding of this communication is strictly prohibited. If you have received this communication
in error, please contact the sender immediately and delete it from your system. Thank You.
>>  
>> -- 
>> Indrashish Basu 
>> Graduate Student 
>> Department of Electrical and Computer Engineering 
>> University of Florida
>> 
>  
> -- 
> Indrashish Basu 
> Graduate Student 
> Department of Electrical and Computer Engineering 
> University of Florida
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message