Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D57551026C for ; Thu, 17 Apr 2014 11:24:03 +0000 (UTC) Received: (qmail 3264 invoked by uid 500); 17 Apr 2014 11:23:56 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 3181 invoked by uid 500); 17 Apr 2014 11:23:55 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 3172 invoked by uid 99); 17 Apr 2014 11:23:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Apr 2014 11:23:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of karim.awara@kaust.edu.sa designates 74.125.82.46 as permitted sender) Received: from [74.125.82.46] (HELO mail-wg0-f46.google.com) (74.125.82.46) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Apr 2014 11:23:49 +0000 Received: by mail-wg0-f46.google.com with SMTP id b13so308942wgh.5 for ; Thu, 17 Apr 2014 04:23:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=wpJuBFeG/wLI7C+BejW7bBMvgUglryNFJ3/FnloxrCY=; b=Oz5KPHiTJav3WvR8E1mWkK8sPl5V+A0c/QLiGLDPeVFXy56HYSrzjFNP2fYsFDb3rY gM3XcgzglOMUqSdIpDOcY593BvowsEP/2TGBBBlZxjlyWS+93GiLA/lfE56bQYNntHOe bX3G63y/EZO3PeXa+4M/zXA1eThS6hwndqG8y7cQv6R8am0OEoWPKuLVkUTSTjFnyZcb bjhE8xA9yQnpCZeruTsQV6NoSI23yz/n3q8jqAggmBSZZEDPOghYv1upmgdErzy1+ziv nrIDM+/zrcvPseWvqHSYVKMcbBIaL0EMFiAVnPZ7cZ5wlMKmVlZ9DIE0JyNAzT6jWoek UUHA== X-Gm-Message-State: ALoCoQmuFAIec4QvfyPSPt9icOMMMl+IN/DBmDU/42Ew4wtTC9LYt0x0fVSKX0fX0AVI48G18/i4HMwFJQ0xqgz4pi+HeO0hBOhAoma9wqPut8Sjc9qkHn4= X-Received: by 10.180.160.205 with SMTP id xm13mr4303477wib.22.1397733807630; Thu, 17 Apr 2014 04:23:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.227.232.69 with HTTP; Thu, 17 Apr 2014 04:23:07 -0700 (PDT) In-Reply-To: References: From: Karim Awara Date: Thu, 17 Apr 2014 14:23:07 +0300 Message-ID: Subject: Re: Task or job tracker seems not working? To: user Content-Type: multipart/alternative; boundary=047d7b62494e18370104f73b43d9 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b62494e18370104f73b43d9 Content-Type: text/plain; charset=US-ASCII If cluster is undergoing some networking issue, it means that HDFS shouldn't be working as well right? The cluster is quite free for my job with high specification 18GB memory each machine, quad core. -- Best Regards, Karim Ahmed Awara On Thu, Apr 17, 2014 at 2:18 PM, Nitin Pawar wrote: > you need to allocate sufficient memory to datanodes as well. > Also make sure that none of the network cards on your datanodes have > turned bad. > > Most of the time the error you saw comes when there is heavy utilization > of cluser or it is undergoing some kind of network issue. > > > On Thu, Apr 17, 2014 at 4:37 PM, Karim Awara wrote: > >> >> im setting mapred.child.java.opts to Xmx8G. my dataset im using for >> the mapreduce job is quite small (few hundred megabytes) as well. >> >> -- >> Best Regards, >> Karim Ahmed Awara >> >> >> On Thu, Apr 17, 2014 at 2:01 PM, Nitin Pawar wrote: >> >>> Can you tell us JVM memory allocated to all data nodes? >>> >>> >>> On Thu, Apr 17, 2014 at 4:28 PM, Karim Awara wrote: >>> >>>> Hi, >>>> >>>> I am running a mpreduce job on a cluster of 16 machines. The HDFS is >>>> working normally however, when I ran a mapreduce job, it gives an error: >>>> >>>> Java.io.IOException: Bad connect act with firstBadLink >>>> >>>> although I have all the processes up.. >>>> >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> Karim Ahmed Awara >>>> >>>> ------------------------------ >>>> This message and its contents, including attachments are intended >>>> solely for the original recipient. If you are not the intended recipient or >>>> have received this message in error, please notify me immediately and >>>> delete this message from your computer system. Any unauthorized use or >>>> distribution is prohibited. Please consider the environment before printing >>>> this email. >>> >>> >>> >>> >>> -- >>> Nitin Pawar >>> >> >> >> ------------------------------ >> This message and its contents, including attachments are intended solely >> for the original recipient. If you are not the intended recipient or have >> received this message in error, please notify me immediately and delete >> this message from your computer system. Any unauthorized use or >> distribution is prohibited. Please consider the environment before printing >> this email. >> > > > > -- > Nitin Pawar > -- ------------------------------ This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email. --047d7b62494e18370104f73b43d9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

If cluster is undergoing some networking is= sue, it means that HDFS shouldn't be working as well right? The cluster= is quite free for my job with high specification 18GB memory each machine,= quad core.

--=
Best Regards,
Karim Ahmed Awara


On Thu, Apr 17, 2014 at 2:18 PM, Nitin P= awar <nitinpawar432@gmail.com> wrote:
you need to allocate sufficient memory to datanodes as wel= l.=A0
Also make sure that none of the network cards on your datanodes h= ave turned bad.

Most of the time the error you saw= comes when there is heavy utilization of cluser or it is undergoing some k= ind of network issue.=A0

On Thu, Apr 17, 2014 at 4:37 PM, Karim Awa= ra <karim.awara@kaust.edu.sa> wrote:

im setting= =A0 mapred.child.java.opts to Xmx8G.=A0=A0=A0 my dataset im using for the m= apreduce job is quite small (few hundred megabytes) as well.

--
Best Regards,
Karim Ahmed Awara


On Thu, Apr 17, 2014 at = 2:01 PM, Nitin Pawar <nitinpawar432@gmail.com> wrote:<= br>
Can you tell us JVM memory allocated to all data nodes?=A0=


On Thu, Apr 17, 2014 at 4:28 PM, Karim Awara <karim.awara@kaust.= edu.sa> wrote:
=
Hi,

I am running a mpreduce job on a cluster of 16 = machines.=A0 The HDFS is working normally however, when I ran a mapreduce j= ob, it gives an error:

Java.io.IOException: Bad connect act= with firstBadLink

although I have all the processes up..

<= br>

--
Best Re= gards,
Karim Ahmed Awara

This message and= its contents, including attachments are intended solely for the original r= ecipient. If you are not the intended recipient or have received this messa= ge in error, please notify me immediately and delete this message from your= computer system. Any unauthorized use or distribution is prohibited. Pleas= e consider the environment before printing this email.



--
Nitin Pawar


This message and its content= s, including attachments are intended solely for the original recipient. If= you are not the intended recipient or have received this message in error,= please notify me immediately and delete this message from your computer sy= stem. Any unauthorized use or distribution is prohibited. Please consider t= he environment before printing this email.



--
Nitin = Pawar


This message and its content= s, including attachments are intended solely for the original recipient. If= you are not the intended recipient or have received this message in error,= please notify me immediately and delete this message from your computer sy= stem. Any unauthorized use or distribution is prohibited. Please consider t= he environment before printing this email. --047d7b62494e18370104f73b43d9--