hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuan Gong <xg...@hortonworks.com>
Subject Re: "No FileSystem for scheme: hdfs" when submitting a remote job
Date Tue, 24 Feb 2015 19:03:25 GMT
Have you added hadoop-hdfs-xxxx.jar into your class path ?

From: xeonmailinglist <xeonmailinglist@gmail.com<mailto:xeonmailinglist@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Tuesday, February 24, 2015 at 10:57 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: "No FileSystem for scheme: hdfs" when submitting a remote job


I am trying to submit a remote mapreduce job in Yarn with the configuration below, but I get
the error No FileSystem for scheme: hdfs. The error is so also below. Why this is happening?

         String host = "";

           Configuration conf = job.getConfiguration();
           // this should be like defined in your yarn-site.xml
           conf.set("yarn.resourcemanager.address", host + ":50001");

            // framework is now "yarn", should be defined like this in mapred-site.xm
            conf.set("mapreduce.framework.name", "yarn");

            // like defined in hdfs-site.xml
            conf.set("fs.defaultFS", "hdfs://" + host + ":9000");

            for (Path inputPath : inputPaths)
                FileInputFormat.addInputPath(job, new Path(conf.get("fs.defaultFS") + "/"
+ inputPath.toString()));

            FileOutputFormat.setOutputPath(job, new Path(conf.get("fs.defaultFS") + "/" +


Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:518)
    at org.apache.hadoop.mapred.examples.JobExecution.submit(JobExecution.java:169)
    at org.apache.hadoop.mapred.examples.WordCount.main(WordCount.java:101)


View raw message