hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Yarn Writing ApplicationMaster, Client Exception "URISyntaxException: Expected scheme name at index 0"
Date Wed, 27 Mar 2013 21:11:07 GMT
That seems like a bug - we should at least throw a better error.

On Mar 25, 2013, at 5:29 PM, Harsh J wrote:

> YARN does not seem to be checking for a fully qualified path when you
> pass it yours and ends up breaking. The problem is easily reproducible
> with the two transforming calls from ConverterUtils.
> 
> Transform the jarPath to a fully qualified one like so, before using
> it anywhere:
> 
> Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR);
> jarPath = fs.makeQualified(jarPath);
> 
> And the problem should disappear.
> 
> 
> On Mon, Mar 25, 2013 at 10:57 PM, blah blah <tmp5330@gmail.com> wrote:
>> Hi
>> 
>> I am trying to create my own Application Master. I have followed this
>> tutorial
>> http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
>> However I have problem with reading AM jar as resource at the container.
>> Code for passing AM.jar
>> 
>> Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR);
>> fs.copyFromLocalFile(new Path("/local/src/AM.jar"), jarPath); // VALIDATED
>> jar is in HDFS under correct PATH
>> FileStatus jarStatus = fs.getFileStatus(jarPath);
>> LocalResource amJarRsrc = Records.newRecord(LocalResource.class);
>> amJarRsrc.setType(LocalResourceType.FILE);
>> amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
>> amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath));
>> amJarRsrc.setTimestamp(jarStatus.getModificationTime());
>> amJarRsrc.setSize(jarStatus.getLen());
>> localResources.put("AppMaster.jar",  amJarRsrc);
>> amContainer.setLocalResources(localResources);
>> 
>> Error logs (nodeManager.log)
>> 
>> INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
>> Application application_1364219323374_0016 transitioned from INITING to
>> RUNNING
>> INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
>> Got exception parsing AppMaster.jar and value resource {, port: -1, file:
>> "/Working_HDFS_DIR/application_1364219323374_0016/AM.jar", }, size: 13940,
>> timestamp: 1364230436600, type: FILE, visibility: APPLICATION,
>> 2013-03-25 17:53:57,391 WARN
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
>> Failed to parse resource-request
>> java.net.URISyntaxException: Expected scheme name at index 0:
>> :///Working_HDFS_DIR/application_1364219323374_0016/AM.jar
>> at java.net.URI$Parser.fail(URI.java:2810)
>> at java.net.URI$Parser.failExpecting(URI.java:2816)
>> at java.net.URI$Parser.parse(URI.java:3008)
>> at java.net.URI.<init>(URI.java:735)
>> at
>> org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:70)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:501)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:472)
>> at
>> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382)
>> at
>> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
>> at
>> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
>> at
>> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:828)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:71)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:556)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:549)
>> at
>> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:130)
>> at
>> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
>> at java.lang.Thread.run(Thread.java:662)
>> 
>> Where am I making the mistake?
>> 
>> regards
>> tmp
> 
> 
> 
> -- 
> Harsh J

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



Mime
View raw message