hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karim Awara <karim.aw...@kaust.edu.sa>
Subject Re: Task or job tracker seems not working?
Date Thu, 17 Apr 2014 11:23:07 GMT
If cluster is undergoing some networking issue, it means that HDFS
shouldn't be working as well right? The cluster is quite free for my job
with high specification 18GB memory each machine, quad core.

--
Best Regards,
Karim Ahmed Awara


On Thu, Apr 17, 2014 at 2:18 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:

> you need to allocate sufficient memory to datanodes as well.
> Also make sure that none of the network cards on your datanodes have
> turned bad.
>
> Most of the time the error you saw comes when there is heavy utilization
> of cluser or it is undergoing some kind of network issue.
>
>
> On Thu, Apr 17, 2014 at 4:37 PM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>
>>
>> im setting  mapred.child.java.opts to Xmx8G.    my dataset im using for
>> the mapreduce job is quite small (few hundred megabytes) as well.
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>>
>> On Thu, Apr 17, 2014 at 2:01 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:
>>
>>> Can you tell us JVM memory allocated to all data nodes?
>>>
>>>
>>> On Thu, Apr 17, 2014 at 4:28 PM, Karim Awara <karim.awara@kaust.edu.sa>wrote:
>>>
>>>> Hi,
>>>>
>>>> I am running a mpreduce job on a cluster of 16 machines.  The HDFS is
>>>> working normally however, when I ran a mapreduce job, it gives an error:
>>>>
>>>> Java.io.IOException: Bad connect act with firstBadLink
>>>>
>>>> although I have all the processes up..
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Karim Ahmed Awara
>>>>
>>>> ------------------------------
>>>> This message and its contents, including attachments are intended
>>>> solely for the original recipient. If you are not the intended recipient
or
>>>> have received this message in error, please notify me immediately and
>>>> delete this message from your computer system. Any unauthorized use or
>>>> distribution is prohibited. Please consider the environment before printing
>>>> this email.
>>>
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>>
>
>
>
> --
> Nitin Pawar
>

-- 

------------------------------
This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

Mime
View raw message