Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6BB50FA38 for ; Mon, 25 Mar 2013 17:28:26 +0000 (UTC) Received: (qmail 29305 invoked by uid 500); 25 Mar 2013 17:28:21 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 29171 invoked by uid 500); 25 Mar 2013 17:28:21 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 29163 invoked by uid 99); 25 Mar 2013 17:28:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Mar 2013 17:28:21 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tmp5330@gmail.com designates 209.85.215.42 as permitted sender) Received: from [209.85.215.42] (HELO mail-la0-f42.google.com) (209.85.215.42) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Mar 2013 17:28:14 +0000 Received: by mail-la0-f42.google.com with SMTP id fe20so11906374lab.29 for ; Mon, 25 Mar 2013 10:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=KIv3X7AxmFwDl6sLGspAB2GiRrVYsuV9ytXgJKq9sYM=; b=tYk0CuW2N9e/daFkpbI58d6HRJIAbcAnuTAHQc+ufxY//NhMlv10MFXfv9Wh4YI0ai YkD6Pwp0Y+HNb/+E1rK8LGsB+afyw4aG23FiDzoxU1ieztqRdnPoqxFDS0/TUUypPmij 4azb3WiAFyApwcg0Y1yPxTHXmKmF5C3hekG7JlSUBIwPiW8O250m12iVIasqy16+B6Hu 9pgWojFKi5EVc3TPZRZJauwPm6OHEqkoOyLpN3oa55UICiabQVGQcrx4u5i/zcnFBayI cD76J1TC7OQ1q6NBnADMIY8jLoWCDNAo46JjZQsb03YhhILZP+O35ZGHvPaosRvIvVVI y2vA== MIME-Version: 1.0 X-Received: by 10.152.48.113 with SMTP id k17mr6564598lan.29.1364232473422; Mon, 25 Mar 2013 10:27:53 -0700 (PDT) Received: by 10.114.57.239 with HTTP; Mon, 25 Mar 2013 10:27:53 -0700 (PDT) Date: Mon, 25 Mar 2013 18:27:53 +0100 Message-ID: Subject: Yarn Writing ApplicationMaster, Client Exception "URISyntaxException: Expected scheme name at index 0" From: blah blah To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=bcaec5524414f82f4204d8c31f43 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5524414f82f4204d8c31f43 Content-Type: text/plain; charset=ISO-8859-1 Hi I am trying to create my own Application Master. I have followed this tutorial http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html However I have problem with reading AM jar as resource at the container. Code for passing AM.jar Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR); fs.copyFromLocalFile(new Path("/local/src/AM.jar"), jarPath); // VALIDATED jar is in HDFS under correct PATH FileStatus jarStatus = fs.getFileStatus(jarPath); LocalResource amJarRsrc = Records.newRecord(LocalResource.class); amJarRsrc.setType(LocalResourceType.FILE); amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION); amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath)); amJarRsrc.setTimestamp(jarStatus.getModificationTime()); amJarRsrc.setSize(jarStatus.getLen()); localResources.put("AppMaster.jar", amJarRsrc); amContainer.setLocalResources(localResources); Error logs (nodeManager.log) INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1364219323374_0016 transitioned from INITING to RUNNING INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Got exception parsing AppMaster.jar and value resource {, port: -1, file: "/Working_HDFS_DIR/application_1364219323374_0016/AM.jar", }, size: 13940, timestamp: 1364230436600, type: FILE, visibility: APPLICATION, 2013-03-25 17:53:57,391 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Failed to parse resource-request java.net.URISyntaxException: Expected scheme name at index 0: :///Working_HDFS_DIR/application_1364219323374_0016/AM.jar at java.net.URI$Parser.fail(URI.java:2810) at java.net.URI$Parser.failExpecting(URI.java:2816) at java.net.URI$Parser.parse(URI.java:3008) at java.net.URI.(URI.java:735) at org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:70) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.(LocalResourceRequest.java:46) at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:501) at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:472) at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445) at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:828) at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:71) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:556) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:549) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:130) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77) at java.lang.Thread.run(Thread.java:662) Where am I making the mistake? regards tmp --bcaec5524414f82f4204d8c31f43 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi

I am trying to create my own Application Master. I ha= ve followed this tutorial=A0http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoo= p-yarn-site/WritingYarnApplications.html=A0However I have problem with = reading AM jar as resource at the container. Code for passing AM.jar

Path jarPath =3D new Path("/Working_HDFS_DIR/= "+ appId +"/"+AM_JAR);
fs.copyFromLocalFile(new Pa= th("/local/src/AM.jar"), jarPath); // VALIDATED jar is in HDFS un= der correct PATH
FileStatus jarStatus =3D fs.getFileStatus(jarPath);
LocalRes= ource amJarRsrc =3D Records.newRecord(LocalResource.class);
amJar= Rsrc.setType(LocalResourceType.FILE);
amJarRsrc.setVisibility(Loc= alResourceVisibility.APPLICATION);
amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath));
amJarRsrc.setTimestamp(jarStatus.getModificationTime());
a= mJarRsrc.setSize(jarStatus.getLen());
localResources.put("Ap= pMaster.jar", =A0amJarRsrc);
amContainer.setLocalResources(localResources);

Error logs (nodeManager.log)

INFO org= .apache.hadoop.yarn.server.nodemanager.containermanager.application.Applica= tion: Application application_1364219323374_0016 transitioned from INITING = to RUNNING
INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.contai= ner.Container: Got exception parsing AppMaster.jar and value resource {, po= rt: -1, file: "/Working_HDFS_DIR/application_1364219323374_0016/AM.jar= ", }, size: 13940, timestamp: 1364230436600, type: FILE, visibility: A= PPLICATION,=A0
2013-03-25 17:53:57,391 WARN org.apache.hadoop.yarn.server.nodemanager= .containermanager.container.Container: Failed to parse resource-request
java.net.URISyntaxException: Expected scheme name at index 0: :///Wo= rking_HDFS_DIR/application_1364219323374_0016/AM.jar
at ja= va.net.URI$Parser.fail(URI.java:2810)
at java.net.URI$Parser.failExpecting(= URI.java:2816)
at ja= va.net.URI$Parser.parse(URI.java:3008)
at java.net.URI.<init>(URI.jav= a:735)
at or= g.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.= java:70)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.loca= lizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
at or= g.apache.hadoop.yarn.server.nodemanager.containermanager.container.Containe= rImpl$RequestResourcesTransition.transition(ContainerImpl.java:501)
at org.apa= che.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl= $RequestResourcesTransition.transition(ContainerImpl.java:472)
at org.apach= e.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(St= ateMachineFactory.java:382)
at or= g.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFac= tory.java:299)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$3= 00(StateMachineFactory.java:43)
at or= g.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTrans= ition(StateMachineFactory.java:445)
at org.apache.hadoop.yarn.server.nodema= nager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:82= 8)
at or= g.apache.hadoop.yarn.server.nodemanager.containermanager.container.Containe= rImpl.handle(ContainerImpl.java:71)
at org.apache.hadoop.yarn.server.nodema= nager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle= (ContainerManagerImpl.java:556)
at or= g.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerIm= pl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:549)
at org.apa= che.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:130)
at= org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77= )
at ja= va.lang.Thread.run(Thread.java:662)

Where am= I making the mistake?

regards
tmp
--bcaec5524414f82f4204d8c31f43--