hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukas Kairies <lukas.xtree...@googlemail.com>
Subject Invalide URI in job start
Date Mon, 27 Jan 2014 10:05:05 GMT
Hello,

I try to use XtreemFS as an alternative file system for Hadoop 2.x. 
There is an existing FileSystem implementation for Hadoop 1.x that works 
fine. First think I did was to implement a DelegateToFileSystem subclass 
to provide an AbstractFileSystem implementation for XtreemFS (just 
constructors that use the FileSystem implementation). When I start the 
wordcount example application I get the following Exception on the 
NodeManager:

2014-01-20 14:18:19,349 WARN 
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: 
Failed to parse resource-request
java.net.URISyntaxException: Expected scheme name at index 0: 
:///tmp/hadoop-yarn/staging/lkairies/.staging/job_1390223418764_0004/job.jar
         at java.net.URI$Parser.fail(URI.java:2829)
         at java.net.URI$Parser.failExpecting(URI.java:2835)
         at java.net.URI$Parser.parse(URI.java:3027)
         at java.net.URI.<init>(URI.java:753)
         at 
org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:80)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:529)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:497)
         at 
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
         at 
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
         at 
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
         at 
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:864)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:73)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:815)
         at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:808)
         at 
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:134)
         at 
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:81)
         at java.lang.Thread.run(Thread.java:724)

Additionally the following is printed on the console:

14/01/27 11:02:14 INFO input.FileInputFormat: Total input paths to 
process : 1
14/01/27 11:02:14 INFO mapreduce.JobSubmitter: number of splits:1
14/01/27 11:02:15 INFO Configuration.deprecation: user.name is 
deprecated. Instead, use mapreduce.job.user.name
14/01/27 11:02:15 INFO Configuration.deprecation: mapred.jar is 
deprecated. Instead, use mapreduce.job.jar
14/01/27 11:02:15 INFO Configuration.deprecation: 
mapred.output.value.class is deprecated. Instead, use 
mapreduce.job.output.value.class
14/01/27 11:02:15 INFO Configuration.deprecation: 
mapreduce.combine.class is deprecated. Instead, use 
mapreduce.job.combine.class
14/01/27 11:02:15 INFO Configuration.deprecation: mapreduce.map.class is 
deprecated. Instead, use mapreduce.job.map.class
14/01/27 11:02:15 INFO Configuration.deprecation: mapred.job.name is 
deprecated. Instead, use mapreduce.job.name
14/01/27 11:02:15 INFO Configuration.deprecation: mapreduce.reduce.class 
is deprecated. Instead, use mapreduce.job.reduce.class
14/01/27 11:02:15 INFO Configuration.deprecation: mapred.input.dir is 
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/01/27 11:02:15 INFO Configuration.deprecation: mapred.output.dir is 
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/01/27 11:02:15 INFO Configuration.deprecation: mapred.map.tasks is 
deprecated. Instead, use mapreduce.job.maps
14/01/27 11:02:15 INFO Configuration.deprecation: 
mapred.output.key.class is deprecated. Instead, use 
mapreduce.job.output.key.class
14/01/27 11:02:15 INFO Configuration.deprecation: mapred.working.dir is 
deprecated. Instead, use mapreduce.job.working.dir
14/01/27 11:02:15 INFO mapreduce.JobSubmitter: Submitting tokens for 
job: job_1390816735288_0001
14/01/27 11:02:15 INFO impl.YarnClientImpl: Submitted application 
application_1390816735288_0001 to ResourceManager at /0.0.0.0:8032
14/01/27 11:02:15 INFO mapreduce.Job: The url to track the job: 
http://ludiwg:8088/proxy/application_1390816735288_0001/
14/01/27 11:02:15 INFO mapreduce.Job: Running job: job_1390816735288_0001
14/01/27 11:02:19 INFO mapreduce.Job: Job job_1390816735288_0001 running 
in uber mode : false
14/01/27 11:02:19 INFO mapreduce.Job:  map 0% reduce 0%
14/01/27 11:02:19 INFO mapreduce.Job: Job job_1390816735288_0001 failed 
with state FAILED due to: Application application_1390816735288_0001 
failed 2 times due to AM Container for 
appattempt_1390816735288_0001_000002 exited with  exitCode: -1000 due 
to: .Failing this attempt.. Failing the application.
14/01/27 11:02:19 INFO mapreduce.Job: Counters: 0


The job files are created in XtreemFS. After a lot of debugging I still 
did not find the problem.

Any idea how to fix this?

Regards,
Lukas

Mime
View raw message