hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Basu,Indrashish" <indrash...@ufl.edu>
Subject Re: Error while running Hadoop Source Code
Date Wed, 06 Nov 2013 22:50:45 GMT
  

Can anyone please assist regarding this ? 

Thanks in advance


Regards, 

Indra 

On Wed, 06 Nov 2013 09:50:02 -0500, Basu,Indrashish
wrote: 

> Hi Vinod, 
> 
> Thanks for your help regarding this. I
checked the task logs, this is what it is giving as output. 
> 
>
2013-11-06 06:40:05,541 INFO org.apache.hadoop.mapred.JvmManager: In
JvmRunner constructed JVM ID: jvm_201311060636_0001_m_1100862588 
> 
>
2013-11-06 06:40:05,553 INFO org.apache.hadoop.mapred.JvmManager: JVM
Runner jvm_201311060636_0001_m_1100862588 spawned. 
> 
> 2013-11-06
06:40:05,650 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner
constructed JVM ID: jvm_201311060636_0001_m_-1960039766 
> 
> 2013-11-06
06:40:05,651 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner
jvm_201311060636_0001_m_-1960039766 spawned. 
> 
> 2013-11-06
06:40:07,496 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID:
jvm_201311060636_0001_m_1100862588 given task: attempt_201311060 
> 
>
636_0001_m_000001_3 
> 
> 2013-11-06 06:40:07,618 INFO
org.apache.hadoop.mapred.TaskTracker: JVM with ID:
jvm_201311060636_0001_m_-1960039766 given task: attempt_20131106 
> 
>
0636_0001_m_000000_3 
> 
> 2013-11-06 06:40:08,013 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from status:
false 
> 
> 2013-11-06 06:40:08,014 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

> 
> 2013-11-06 06:40:08,015 INFO org.apache.hadoop.mapred.TaskTracker:
running on gpu xx from status: false 
> 
> 2013-11-06 06:40:08,015 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

> 
> 2013-11-06 06:40:09,361 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311060636_0001_m_000001_3 0.0% 
> 
> 2013-11-06 06:40:09,735
INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311060636_0001_m_000000_3 0.0% 
> 
> 2013-11-06 06:40:10,018
INFO org.apache.hadoop.mapred.JvmManager: JVM :
jvm_201311060636_0001_m_1100862588 exited. Number of tasks it ran: 0 
>

> 2013-11-06 06:40:11,021 INFO org.apache.hadoop.mapred.JvmManager: JVM
: jvm_201311060636_0001_m_-1960039766 exited. Number of tasks it ran: 0

> 
> 2013-11-06 06:40:11,442 INFO org.apache.hadoop.mapred.TaskTracker:
running on gpu xx from status: false 
> 
> 2013-11-06 06:40:11,442 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

> 
> 2013-11-06 06:40:11,443 INFO org.apache.hadoop.mapred.TaskTracker:
running on gpu xx from status: false 
> 
> 2013-11-06 06:40:11,443 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

> 
> 2013-11-06 06:40:13,021 INFO org.apache.hadoop.mapred.TaskRunner:
attempt_201311060636_0001_m_000001_3 done; removing files. 
> 
>
2013-11-06 06:40:13,037 INFO org.apache.hadoop.mapred.TaskTracker:
addCPUFreeSlot : current free slots : 3 
> 
> 2013-11-06 06:40:14,025
INFO org.apache.hadoop.mapred.TaskRunner:
attempt_201311060636_0001_m_000000_3 done; removing files. 
> 
>
2013-11-06 06:40:14,028 INFO org.apache.hadoop.mapred.TaskTracker:
addCPUFreeSlot : current free slots : 4 
> 
> 2013-11-06 06:40:14,476
INFO org.apache.hadoop.mapred.TaskTracker: running on gpu xx from
status: false 
> 
> 2013-11-06 06:40:14,477 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

> 
> 2013-11-06 06:40:14,477 INFO org.apache.hadoop.mapred.TaskTracker:
running on gpu xx from status: false 
> 
> 2013-11-06 06:40:14,477 INFO
org.apache.hadoop.mapred.TaskTracker: running on gpu xx from task: false

> 
> 2013-11-06 06:40:14,894 INFO org.apache.hadoop.mapred.TaskTracker:
running on gpu : false 
> 
> 2013-11-06 06:40:14,900 INFO
org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask):
attempt_201311060636_0001_m_000002_0 task's 
> 
> state:UNASSIGNED 
> 
>
2013-11-06 06:40:14,902 INFO org.apache.hadoop.mapred.TaskTracker:
Trying to launch : attempt_201311060636_0001_m_000002_0 on CPU 
> 
>
2013-11-06 06:40:14,904 INFO org.apache.hadoop.mapred.TaskTracker: In
TaskLauncher, current free CPU slots : 4 and trying to launch
attempt_2013 
> 
> 11060636_0001_m_000002_0 
> 
> 2013-11-06
06:40:14,909 INFO org.apache.hadoop.mapred.TaskTracker: Received
KillTaskAction for task: attempt_201311060636_0001_m_000000_3 
> 
>
2013-11-06 06:40:14,920 INFO org.apache.hadoop.mapred.TaskTracker:
Received KillTaskAction for task: attempt_201311060636_0001_m_000001_3

> 
> 2013-11-06 06:40:15,161 INFO org.apache.hadoop.mapred.JvmManager:
In JvmRunner constructed JVM ID: jvm_201311060636_0001_m_164532908 
> 
>
2013-11-06 06:40:15,162 INFO org.apache.hadoop.mapred.JvmManager: JVM
Runner jvm_201311060636_0001_m_164532908 spawned. 
> 
> 2013-11-06
06:40:17,216 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID:
jvm_201311060636_0001_m_164532908 given task: attempt_2013110606 
> 
>
36_0001_m_000002_0 
> 
> Regards, 
> 
> Indrashish 
> 
> On Tue, 5 Nov
2013 10:09:36 -0800, Vinod Kumar Vavilapalli wrote: 
> 
>> It seems like
your pipes mapper is exiting before consuming all the input. Did you
check the task-logs on the web UI? 
>> 
>> Thanks, 
>> +Vinod 
>> 
>> On
Nov 5, 2013, at 7:25 AM, Basu,Indrashish wrote: 
>> 
>>> Hi,
>>> 
>>>
Can anyone kindly assist on this ?
>>> 
>>> Regards,
>>> Indrashish
>>>

>>> On Mon, 04 Nov 2013 10:23:23 -0500, Basu,Indrashish wrote:
>>>

>>>> Hi All,
>>> 
>>>> Any update on the below post ?
>>>
*WordCountInputFor
>>> ich has a very specific input split probably be
recoded *to* use libhdfs
>>> hat is more C++ example is using local file
reads, it will only work on 
>>> 
>>>> eft: 5px; width: 100%;">if you
have nfs or something working across
>>> " I would need some more light
on the ab
>>> 
>>>> 2px solid; margin-left: 5px; width: 100%;">could
elaborate
>>> the same as what needs to be done exactly. using Hadoop.
in advance.
>>> 0ff 2px solid; margin-left: 5px; width: 100%;">Regards,
On
>>> 013 20:00:10 -0400, Basu,Indrashish wrote: 
>>> 
>>>> ">Hi,
>>>
eft: #1010ff 2px solid; margin-left: 5px; width: 100%;"> rmr: cannot
r
>>> 
>>>> px; border-left: #
>>> solid; margin-left: 5px; width:
100%;"> : 5px; widt
>>> sses may not be found. See JobConf(Class) or
JobConf#setJar(String). 13/10/31 13:43:12 INFO mapred.FileInputFormat:
Total input pat
>>> uote> process : 1
>>> px; border-left: #1010ff 2px
solid; margin-left: 5px; width: 100%;">13/10/31 13:43:13 INFO mapr
>>>

>>>> -left: 5px; border-left: #1010ff 2px solid; margin-left: 5px;
width: 100%;"> 13/
>>> 4 INFO mapred.JobClient: map 0% reduce 0%
attempt_201310311320_0001_m_000000_0, Status : FAILED
java.io.IOException: pipe child exception
>>> id; margin-left: 5px;
width: 100%;"> at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:191)
left: 5px; width: 100%;"> 
>>>> 
>>>>> at
>>> g-left: 5px; border-left:
#1010ff 2px solid; margin-left: 5px; width: 100%;"> at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>>> r-left:
#1010ff 2px solid; margin-left: 5px; width: 100%;"> at
org.apache.hadoop.mapred.Child.main(Child.java:170) 
>>> 
>>>> "> 
>>>>

>>>>> Caused by: java.net.SocketException: Broken pipe
>>>
le="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px;
width: 100%;"> at
java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) width:
100%;"> at java.io.DataOutputStream.write
>>> ream.java:107) at left:
5px; width: 100%;"> org.apache.hadoop.mapred.pipes.BinaryProtoc
>>>
t(BinaryProtocol.java:333)
org.apache.hadoop.mapred.pipes.BinaryProtocol.mapItem(BinaryProtocol.java:286)
at
>>> g-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px;
width:
100%;">org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:92)
n-left: 5px; width: 100%;"> &n
>>> 3 more attempt_201310
>>> 
>>>> t:
5px; border-left: #1010ff 2px solid; margin-left: 5px; width: 100%;">
'/app/hadoop/tm
>>>
l/taskTracker/archive/10.227.56.195bin/cpu-kmeans2D/cpu-kmeans2D' 
>>>

>>>> '0' < /dev/null 1>>
>>> er-left: #1010ff 2px solid; margin-left:
5px; width:
100%;">/usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/attempt_201310311320_0001_m_000000_0/stdout
ote style="padding-left: 5px; border-left: #1010ff 2px solid;
margin-left: 5px; width: 100%;">2>>
/usr/local/hadoop/hadoop-gpu-0.20.1/bin/../logs/userlogs/
>>>
e="padding-left: 5px; border-left: #1010ff 2px solid; margin-left: 5px;
width: 100%;"> Regards, 
>>> -- 
>>> Indrashish Basu
>>> Graduate
Stu
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>>

>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>>

>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>> 
>>> 
>>>>

>>> 
>>>> 
>>> 
>>>> 
>> 
>> CONFIDENTIALITY NOTICE
>> NOTICE: This
message is intended for the use of the individual or entity to which it
is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the
reader of this message is not the intended recipient, you are hereby
notified that any printing, copying, dissemination, distribution,
disclosure or forwarding of this communication is strictly prohibited.
If you have received this communication in error, please contact the
sender immediately and delete it from your system. Thank You.
> 
> -- 
>

> Indrashish Basu 
> Graduate Student 
> Department of Electrical and
Computer Engineering 
> University of Florida

-- 

Indrashish Basu

Graduate Student 
Department of Electrical and Computer Engineering

University of Florida

  
Mime
View raw message