hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Basu,Indrashish" <indrash...@ufl.edu>
Subject Re: Error while running Hadoop Source Code
Date Sat, 09 Nov 2013 15:58:03 GMT
  

Any updates on this ? 

Regards, 

Indra 

On Thu, 07 Nov 2013
11:28:57 -0500, Basu,Indrashish wrote: 

> Hi Vinod, 
> 
> Please find
the below details requested: 
> 
> A) ALL OF THE TASKTRACKER LOG : 
> 
>
Same what I have provided in the earlier email. 
> 
> B) THE TASK-LOGS :

> 
> SYSLOG : 
> 
> 2013-11-06 16:37:21,331 INFO
org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with
processName=MAP, sessionId=
> 2013-11-06 16:37:22,245 INFO
org.apache.hadoop.mapred.MapTask: numReduceTasks: 2
> 2013-11-06
16:37:22,301 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100
>
2013-11-06 16:37:22,832 INFO org.apache.hadoop.mapred.MapTask: data
buffer = 79691776/99614720
> 2013-11-06 16:37:22,833 INFO
org.apache.hadoop.mapred.MapTask: record buffer = 262144/327680
>
2013-11-06 16:37:23,157 ERROR
org.apache.hadoop.mapred.pipes.BinaryProtocol: java.io.EOFException
> at
java.io.DataInputStream.readByte(DataInputStream.java:267)
> at
org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:298)
>
at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:319)
>
at
org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:114)
>

> 2013-11-06 16:37:23,157 INFO
org.apache.hadoop.mapred.pipes.Application: Aborting because of
java.net.SocketException: Broken pipe
> at
java.net.SocketOutputStream.socketWrite0(Native Method)
> at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
>
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
> at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
>
at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333)
>
at
org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286)
>
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92)
>
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363)
> at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at
org.apache.hadoop.mapred.Child.main(Child.java:170)
> 
> 2013-11-06
16:37:23,284 WARN org.apache.hadoop.mapred.TaskTracker: Error running
child
> java.io.IOException: pipe child exception
> at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:191)
>
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:103)
>
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363)
> at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at
org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by:
java.net.SocketException: Broken pipe
> at
java.net.SocketOutputStream.socketWrite0(Native Method)
> at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
>
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
> at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
>
at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333)
>
at
org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286)
>
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92)
>
... 3 more
> 
> STDERR : 
> 
> Command:0received before authentication.
Exiting.. 
> 
> STDOUT : 
> 
> cmd: [bash, -c, exec
'/app/hadoop/tmp/mapred/local/taskTracker/archive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans2D'
'0' < /dev/null 1>> /usr/lo
>
cal/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/attempt_201311061630_0001_m_000001_0/stdout
2>> /usr/local/hadoop/hadoop-gpu-0.20.1/bin/../log
>
s/userlogs/attempt_201311061630_0001_m_000001_0/stderr]
> deviceID: 
>

> C) SPECIFIC TASKATTEMPT'S TASKATTEMTPID THAT IS FAILING : 
> 
>
attempt_201311061630_0001_m_000000_0 
> 
>
attempt_201311061630_0001_m_000001_0 
> 
>
attempt_201311061630_0001_m_000002_0 
> 
>
attempt_201311061630_0001_m_000000_1 
> 
>
attempt_201311061630_0001_m_000001_1 
> 
>
attempt_201311061630_0001_m_000003_0 
> 
>
attempt_201311061630_0001_m_000000_2 
> 
>
attempt_201311061630_0001_m_000001_2 
> 
>
attempt_201311061630_0001_m_000000_3 
> 
>
attempt_201311061630_0001_m_000001_3 
> 
> Kindly let me know in case of
any more details required. 
> 
> Thanks again for your time and patience
regarding this. 
> 
> Regards, 
> 
> Indrashish. 
> 
> On Wed, 6 Nov
2013 16:58:53 -0800, Vinod Kumar Vavilapalli wrote: 
> 
>> Don't see
anything in the logs that you pasted. 
>> Can you paste the following in
say pastebin? 
>> - All of the TaskTracker log 
>> - The task-logs.
These are syslog, stderr, stdout files for a specific TaskAttempt. 
>> -
And specific TaskAttempt's TaskAttemtpID that is failing. 
>> 
>>
Thanks, 
>> +Vinod 
>> 
>> On Nov 6, 2013, at 2:50 PM, Basu,Indrashish
wrote: 
>> 
>>> Can anyone please assist regarding this ? 
>>> 
>>>
Thanks in advance 
>>> 
>>> Regards, 
>>> 
>>> Indra 
>>> 
>>> On Wed,
06 Nov 2013 09:50:02 -0500, Basu,Indrashish wrote: 
>>> 
>>>> Hi Vinod,

>>>> 
>>>> Thanks for your help regarding this. I checked the task
logs, this is what it is giving as output. 
>>>> 
>>>> 2013-11-06
06:40:05,541 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner
constructed JVM ID: jvm_201311060636_0001_m_1100862588 
>>>> 
>>>>
2013-11-06 06:40:05,553 INFO org.apache.hadoop.mapred.JvmManager: JVM
Runner jvm_201311060636_0001_m_1100862588 spawned. 
>>>> 
>>>>
2013-11-06 06:40:05,650 INFO org.apache.hadoop.mapred.JvmManager: In
JvmRunner constructed JVM ID: jvm_201311060636_0001_m_-1960039766 
>>>>

>>>> 2013-11-06 06:40:05,651 INFO org.apache.hadoop.mapred.JvmManager:
JVM Runner jvm_201311060636_0001_m_-1960039766 spawned. 
>>>> 
>>>>
2013-11-06 06:40:07,496 INFO org.apache.hadoop.mapred.TaskTracker: JVM
with ID: jvm_201311060636_0001_m_1100862588 given task:
attempt_201311060 
>>>> 
>>>> 636_0001_m_000001_3 
>>>> 
>>>> 2013-11-06
06:40:07,618 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID:
jvm_201311060636_0001_m_-1960039766 given task: attempt_20131106 
>>>>

>>>> 0636_0001_m_000000_3 
>>>> 
>>>> 2013-11-06 06:40:08,013 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status:
false 
>>>> 
>>>> 2013-11-06 06:40:08,014 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

>>>> 
>>>> 2013-11-06 06:40:08,015 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status:
false 
>>>> 
>>>> 2013-11-06 06:40:08,015 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

>>>> 
>>>> 2013-11-06 06:40:09,361 INFO
org.apache.hadoop.mapred.TaskTracker:
attempt_201311060636_0001_m_000001_3 0.0% 
>>>> 
>>>> 2013-11-06
06:40:09,735 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311060636_0001_m_000000_3 0.0% 
>>>> 
>>>> 2013-11-06
06:40:10,018 INFO org.apache.hadoop.mapred.JvmManager: JVM :
jvm_201311060636_0001_m_1100862588 exited. Number of tasks it ran: 0

>>>> 
>>>> 2013-11-06 06:40:11,021 INFO
org.apache.hadoop.mapred.JvmManager: JVM :
jvm_201311060636_0001_m_-1960039766 exited. Number of tasks it ran: 0

>>>> 
>>>> 2013-11-06 06:40:11,442 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status:
false 
>>>> 
>>>> 2013-11-06 06:40:11,442 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

>>>> 
>>>> 2013-11-06 06:40:11,443 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status:
false 
>>>> 
>>>> 2013-11-06 06:40:11,443 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

>>>> 
>>>> 2013-11-06 06:40:13,021 INFO
org.apache.hadoop.mapred.TaskRunner:
attempt_201311060636_0001_m_000001_3 done; removing files. 
>>>> 
>>>>
2013-11-06 06:40:13,037 INFO org.apache.hadoop.mapred.TaskTracker:
addCPUFreeSlot : current free slots : 3 
>>>> 
>>>> 2013-11-06
06:40:14,025 INFO org.apache.hadoop.mapred.TaskRunner:
attempt_201311060636_0001_m_000000_3 done; removing files. 
>>>> 
>>>>
2013-11-06 06:40:14,028 INFO org.apache.hadoop.mapred.TaskTracker:
addCPUFreeSlot : current free slots : 4 
>>>> 
>>>> 2013-11-06
06:40:14,476 INFO org.apache.hadoop.mapred.TaskTracker: running on gpu
xx from status: false 
>>>> 
>>>> 2013-11-06 06:40:14,477 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

>>>> 
>>>> 2013-11-06 06:40:14,477 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status:
false 
>>>> 
>>>> 2013-11-06 06:40:14,477 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

>>>> 
>>>> 2013-11-06 06:40:14,894 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu : false 
>>>> 
>>>>
2013-11-06 06:40:14,900 INFO org.apache.hadoop.mapred.TaskTracker:
LaunchTaskAction (registerTask): attempt_201311060636_0001_m_000002_0
task's 
>>>> 
>>>> state:UNASSIGNED 
>>>> 
>>>> 2013-11-06 06:40:14,902
INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch :
attempt_201311060636_0001_m_000002_0 on CPU 
>>>> 
>>>> 2013-11-06
06:40:14,904 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher,
current free CPU slots : 4 and trying to launch attempt_2013 
>>>> 
>>>>
11060636_0001_m_000002_0 
>>>> 
>>>> 2013-11-06 06:40:14,909 INFO
org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task:
attempt_201311060636_0001_m_000000_3 
>>>> 
>>>> 2013-11-06 06:40:14,920
INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for
task: attempt_201311060636_0001_m_000001_3 
>>>> 
>>>> 2013-11-06
06:40:15,161 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner
constructed JVM ID: jvm_201311060636_0001_m_164532908 
>>>> 
>>>>
2013-11-06 06:40:15,162 INFO org.apache.hadoop.mapred.JvmManager: JVM
Runner jvm_201311060636_0001_m_164532908 spawned. 
>>>> 
>>>> 2013-11-06
06:40:17,216 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID:
jvm_201311060636_0001_m_164532908 given task: attempt_2013110606 
>>>>

>>>> 36_0001_m_000002_0 
>>>> 
>>>> Regards, 
>>>> 
>>>> Indrashish

>>>> 
>>>> On Tue, 5 Nov 2013 10:09:36 -0800, Vinod Kumar Vavilapalli
wrote: 
>>>> 
>>>>> It seems like your pipes mapper is exiting before
consuming all the input. Did you check the task-logs on the web UI?

>>>>> 
>>>>> Thanks, 
>>>>> +Vinod 
>>>>> 
>>>>> On Nov 5, 2013, at
7:25 AM, Basu,Indrashish wrote: 
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> Can
anyone kindly assist on this ?
>>>>>> 
>>>>>> Regards,
>>>>>>
Indrashish
>>>>>> 
>>>>>> On Mon, 04 Nov 2013 10:23:23 -0500,
Basu,Indrashish wrote:
>>>>>> 
>>>>>>> Hi All,
>>>>>> 
>>>>>>> Any
update on the below post ?
>>>>>> solution as " The *nopipe* example
needs more documentation. It run with the InputFormat from
>>>>>>
border-left: #1010ff 2px solid; margin-left: 5px; width:
100%;">src/test/org/apache/*hadoop*/mapred/*pipes
>>>>>> 
>>>>>>> x;
width: 100%;">*WordCountInputFormat*.java, wh
>>>>>> y specific input
split fo
>>>>>> 
>>>>>>> left: 5px; border-left: #1010ff 2px so
>>>>>>
eft: 5px; width: 100%;">as the input split and won't work right. The
*nopipe* example should 100%;">probably be recoded *to* use libhdfs
*too*, but that is mo
>>>>>> e> c
>>>>>> 
>>>>>>> g-left: 5px;
border-left: #1010ff 2px solid; margin-left: 5px;
>>>>>> ">C++ example
is using local file reads, it will only work on a cluster s or something
working across the cluster. "
>>>>>> ding-left: 5px; border-left:
#1010ff 2px solid; margin-left: 5px; width: 100%;">I would need some
more ligh
>>>>>> 
>>>>>>> 1010ff 2px solid; margin-left: 5px; width:
100%;">coul
>>>>>> bit about the same as what needs to be done exactly.
using Hadoop. Regards, Indrashi
>>>>>> 
>>>>>>> px; width: 100%;">On
Thu, 31 Oct 2013 20:00:10 -0400, Basu
>>>>>> rote: ote>
>>>>>> 2px
solid; margin-left: 5px; width: 100%;"> rmr: cannot remove out
>>>>>>

>>>>>>> ft: #1010ff 2px
>>>>>> rgin-left: 5px; width: 100%;">
%;">classes
>>>>>> ound. See JobConf(Class) or 1010ff 2px solid;
margin-left: 5px; width: 100%;">13/10/31 13:43:12 INFO
mapred.FileInputFormat: Total input paths to
>>>>>> lockquote> process :
1
>>>>>> ft: #1010ff 2px solid; margin-left: 5px; width: 100%;">13/10/31
13:43:13 INFO mapred.JobClient: Running 
>>>>>> 
>>>>>>> 1010ff 2px
solid; margin-left: 5px; width: 100%;"> 13/10/31 13:43:14 INFO ma
>>>>>>
t: map 0% reduce 0% 
>>>>>> 
>>>>>>> eft: 5px; border-left: #1010ff 2px
solid; margin-left: 5px; width: 100%;">
>>>>>> -left: 5px; width:
100%;">attempt_201310311320_0001_m_000000_0, Status : FAILED
java.io.IOException: pipe child exception
>>>>>> kquote
style="padding-left: 5px; border-left: #1010ff 2px solid; margin-left:
5px; width: 100%;"> at
org.apache.hadoop.mapred.pipes.Application.abort(App
>>>>>> 
>>>>>>> 2px
solid; margin-left: 5px; width: 100%;"> 
>>>>>>> 
>>>>>>>> at
>>>>>>
lockquote style="padding-left: 5px; border-left: #1010ff 2px solid;
margin-left: 5px; width: 100%;"> org.apache.hado
>>>>>> 
>>>>>>>
padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px;
width: 100%;">
>>>>>> width: 100%;"> at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>>>>>> #1010ff
2px solid; margin-left: 5px; width: 100%;"> at 
>>>>>> 
>>>>>>> left:
5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">
C
>>>>>> a.net.SocketException: Broken pipe p; at
java.net.SocketOutputStream.socketWrite0(Native Method)
>>>>>> 010ff 2px
solid; margin-left: 5px; width: 100%;"> at
java.net.SocketOutputStream.
>>>>>> 
>>>>>>> border-left: #1010ff 2px
solid; margin-left: 5px; width: 100%;"> at
>>>>>>
ketOutputStream.write(SocketOutputStream.java:159) at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>>>>>
id; margin-left: 5px; width: 100%;"> at
java.io.BufferedOutputStream.write(BufferedOutputStream.java:126
>>>>>>

>>>>>>> argin-left: 5px; width: 100%;"> at java.io.DataO
>>>>>>
rite(DataOutputStream.java:107)
org.apache.hadoop.mapred.pipes.BinaryProtocol.writeObject(BinaryProtocol.java:333)
at
>>>>>> px; width: 100%;">
org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286)
at
>>>>>> ote style="padding-left: 5px; border-left: #1010ff 2px solid;
margin-left: 5px; width:
100%;">org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92)
'/app/hadoop/tmp/mapred/local/taskTracker/archive/
>>>>>>
bin/cpu-kmeans2D/cpu-kmeans2D' '0' < /dev/null 1>>
>>>>>>
e="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px;
width:
100%;">/usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/attempt_201310311320_0001_m_000000_0/stdout
width: 100%;"> 2>>
/usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/
>>>>>> quote>
Regards, gineering
>>>>>>> University of Florida
>>>>>> 
>>>>>>> 
>>>>>>

>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>

>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>

>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>

>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>

>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>>> 
>>>>> 
>>>>> CONFIDENTIALITY
NOTICE
>>>>> NOTICE: This message is intended for the use of the
individual or entity to which it is addressed and may contain
information that is confidential, privileged and exempt from disclosure
under applicable law. If the reader of this message is not the intended
recipient, you are hereby notified that any printing, copying,
dissemination, distribution, disclosure or forwarding of this
communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete
it from your system. Thank You.
>>>> 
>>>> -- 
>>>> 
>>>> Indrashish
Basu 
>>>> Graduate Student 
>>>> Department of Electrical and Computer
Engineering 
>>>> University of Florida
>>> 
>>> -- 
>>> 
>>> Indrashish
Basu 
>>> Graduate Student 
>>> Department of Electrical and Computer
Engineering 
>>> University of Florida
>> 
>> CONFIDENTIALITY NOTICE
>>
NOTICE: This message is intended for the use of the individual or entity
to which it is addressed and may contain information that is
confidential, privileged and exempt from disclosure under applicable
law. If the reader of this message is not the intended recipient, you
are hereby notified that any printing, copying, dissemination,
distribution, disclosure or forwarding of this communication is strictly
prohibited. If you have received this communication in error, please
contact the sender immediately and delete it from your system. Thank
You.
> 
> -- 
> 
> Indrashish Basu 
> Graduate Student 
> Department of
Electrical and Computer Engineering 
> University of Florida

--


Indrashish Basu 
Graduate Student 
Department of Electrical and
Computer Engineering 
University of Florida

  
Mime
View raw message