Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2C94C107D6 for ; Wed, 8 May 2013 17:01:01 +0000 (UTC) Received: (qmail 15307 invoked by uid 500); 8 May 2013 17:00:56 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 15077 invoked by uid 500); 8 May 2013 17:00:56 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 15066 invoked by uid 99); 8 May 2013 17:00:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 May 2013 17:00:55 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of 3m.mustaqeem@gmail.com designates 209.85.160.48 as permitted sender) Received: from [209.85.160.48] (HELO mail-pb0-f48.google.com) (209.85.160.48) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 May 2013 17:00:51 +0000 Received: by mail-pb0-f48.google.com with SMTP id ma3so1348656pbc.35 for ; Wed, 08 May 2013 10:00:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=bmlAEFHtgfxt6k+vsuqmaLmpVTq0mKodaFcy9sRIVN4=; b=A+kjIsJ5vZIwaOpg6UUyjeRybx1KpcXbPT+tDhcgJY+OErSfppfmQRLBPqHQSx63dY S3FY8GTFIWxiekqxfIRKDmETpyhSdX0pzL418GchT/n9LH1+qW+xxl17Ggu6/0jLxDf7 yyYVPuVN0pywD6b/aL1SFBv8Dv+C6R6MZrkn/hVY5gAJoy1DR1TpQGI9I/fpXhYdkLQl gmynDmyXQ+svsETSpVD2y8cV7Vy+wlxSSiloSA0c2cvnIJBOGYexy76AtFFOsCgtXG05 EXPeaQafZI2ELKDPtFy7aHD8Ta+L1aaRGMVZ1t1Cp0eskBWbyZ7+gUralQTCK92VbHD9 1AFA== X-Received: by 10.66.177.177 with SMTP id cr17mr8905951pac.212.1368032431538; Wed, 08 May 2013 10:00:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.70.101.132 with HTTP; Wed, 8 May 2013 10:00:11 -0700 (PDT) In-Reply-To: References: From: Mohammad Mustaqeem <3m.mustaqeem@gmail.com> Date: Wed, 8 May 2013 22:30:11 +0530 Message-ID: Subject: Re: Rack Aware Hadoop cluster To: user Content-Type: multipart/alternative; boundary=047d7bdc7a5e1f88b704dc37df6f X-Virus-Checked: Checked by ClamAV on apache.org --047d7bdc7a5e1f88b704dc37df6f Content-Type: text/plain; charset=ISO-8859-1 @chris, I have test it outside. It is working fine. On Wed, May 8, 2013 at 7:48 PM, Leonid Fedotov wrote: > Error in script. > > > On Wed, May 8, 2013 at 7:11 AM, Chris Embree wrote: > >> Your script has an error in it. Please test your script using both IP >> Addresses and Names, outside of hadoop. >> >> >> On Wed, May 8, 2013 at 10:01 AM, Mohammad Mustaqeem < >> 3m.mustaqeem@gmail.com> wrote: >> >>> I have done this and found following error in log - >>> >>> >>> 2013-05-08 18:53:45,221 WARN org.apache.hadoop.net.ScriptBasedMapping: Exception running /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh 127.0.0.1 >>> org.apache.hadoop.util.Shell$ExitCodeException: /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh: 8: /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh: Syntax error: "(" unexpected (expecting "done") >>> >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:202) >>> at org.apache.hadoop.util.Shell.run(Shell.java:129) >>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:322) >>> at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResolveCommand(ScriptBasedMapping.java:241) >>> at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.resolve(ScriptBasedMapping.java:179) >>> at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:119) >>> at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.resolveNetworkLocation(DatanodeManager.java:454) >>> at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:713) >>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3459) >>> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:881) >>> at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90) >>> at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:18295) >>> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454) >>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014) >>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735) >>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:415) >>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441) >>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729) >>> 2013-05-08 18:53:45,223 ERROR org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: The resolve call returned null! Using /default-rack for host [127.0.0.1] >>> >>> >>> >>> On Wed, May 8, 2013 at 7:18 PM, Leonid Fedotov >> > wrote: >>> >>>> You can put this parameter to core-site.xml or hdfs-site.xml >>>> It both parsed during the HDFS startup. >>>> >>>> Leonid >>>> >>>> >>>> On Wed, May 8, 2013 at 6:43 AM, Mohammad Mustaqeem < >>>> 3m.mustaqeem@gmail.com> wrote: >>>> >>>>> Hello everyone, >>>>> I was searching for how to make the hadoop cluster rack-aware and >>>>> I find out from here >>>>> http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/ClusterSetup.html#Hadoop_Rack_Awareness that >>>>> we can do this by giving property of "topology.script.file.name". But >>>>> here it is not written where to put this >>>>> >>>>> topology.script.file.name >>>>> >>>>> /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh >>>>> >>>>> >>>>> Means in which configuration file. >>>>> I am using hadoop-2.0.3-alpha. >>>>> >>>>> >>>>> -- >>>>> *With regards ---* >>>>> *Mohammad Mustaqeem*, >>>>> M.Tech (CSE) >>>>> MNNIT Allahabad >>>>> 9026604270 >>>>> >>>>> >>>>> >>>> >>> >>> >>> -- >>> *With regards ---* >>> *Mohammad Mustaqeem*, >>> M.Tech (CSE) >>> MNNIT Allahabad >>> 9026604270 >>> >>> >>> >> > -- *With regards ---* *Mohammad Mustaqeem*, M.Tech (CSE) MNNIT Allahabad 9026604270 --047d7bdc7a5e1f88b704dc37df6f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
@chris, I have test it outside. It is working fine.
<= div class=3D"gmail_extra">

On Wed, May 8,= 2013 at 7:48 PM, Leonid Fedotov <lfedotov@hortonworks.com><= /span> wrote:
Error in script.


On Wed, May 8, 2013 at 7:11 AM, Chris Embree <cembree@gmail.com> wrote:
Your script has an error in= it. =A0Please test your script using both IP Addresses and Names, outside = of hadoop.


On = Wed, May 8, 2013 at 10:01 AM, Mohammad Mustaqeem <3m.mustaqeem@gmail.= com> wrote:
I have done this and found = following error in log -=A0


2013-05-08 18:53:45,221 WARN org.apache.hadoop.net.ScriptBasedMapping: Exce=
ption running /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rac=
k.sh 127.0.0.1=20
org.apache.hadoop.util.Shell$ExitCodeException: /home/mustaqeem/development=
/hadoop-2.0.3-alpha/etc/hadoop/rack.sh: 8: /home/mustaqeem/development/hado=
op-2.0.3-alpha/etc/hadoop/rack.sh: Syntax error: "(" unexpected (=
expecting "done")

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:202)
	at org.apache.hadoop.util.Shell.run(Shell.java:129)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:32=
2)
	at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResol=
veCommand(ScriptBasedMapping.java:241)
	at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.resolve(=
ScriptBasedMapping.java:179)
	at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitc=
hMapping.java:119)
	at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.resolveNe=
tworkLocation(DatanodeManager.java:454)
	at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerD=
atanode(DatanodeManager.java:713)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FS=
Namesystem.java:3459)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatano=
de(NameNodeRpcServer.java:881)
	at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorP=
B.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90)
	at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodePr=
otocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:18295)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(=
ProtobufRpcEngine.java:454)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati=
on.java:1441)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
2013-05-08 18:53:45,223 ERROR org.apache.hadoop.hdfs.server.blockmanagement=
.DatanodeManager: The resolve call returned null! Using /default-rack for h=
ost [127.0.0.1]


On Wed, May 8, 2013 at 7:18 PM, Leonid Fedotov <lfedotov@hortonwork= s.com> wrote:
You can put this parameter to core-site.xml or hdfs-site.x= ml
It both parsed during the HDFS startup.

Leonid


On Wed, May 8, 2013 at 6:43 AM, Mohammad Mustaqeem <3m.mustaqeem@gmai= l.com> wrote:
Hello everyone,
=A0 =A0 I was searching for how to mak= e the hadoop cluster rack-aware and I find out from here=A0http://hadoop.apach= e.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/ClusterSetup.html= #Hadoop_Rack_Awareness=A0that we can do this by giving property of &quo= t;topology.= script.file.name". But here it is not written where to put = this=A0
<property>
=A0 =A0 =A0 =A0 <name>topology.script.file.name</name>
=A0 = =A0 =A0 =A0 <value>/home= /mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh</value><= /div>
</property>=A0

Means in which configura= tion file.
I am using hadoop-2.0.3-alpha.


--
With regards ---
Mohammad Mustaqee= m,
M= .Tech (CSE)
MNNIT Allahabad





--
=
With regards ---
= Mohammad Mustaqeem,
M.Tech (CSE)
MNNIT Allahabad

<= br>





--
=
With regards ---
= Mohammad Mustaqeem,
M.Tech (CSE)
MNNIT Allahabad
9026604270

--047d7bdc7a5e1f88b704dc37df6f--