Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of xeonmailinglist@gmail.com
 designates 74.125.82.171 as permitted sender)
Message-ID: <54ECC991.1060702@gmail.com>
Date: Tue, 24 Feb 2015 18:57:21 +0000
From: xeonmailinglist <xeonmailinglist@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: user@hadoop.apache.org
Subject: "No FileSystem for scheme: hdfs" when submitting a remote job
Content-Type: multipart/alternative;
 boundary="------------000709030004030000060703"

This is a multi-part message in MIME format.
--------------000709030004030000060703
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

I am trying to submit a remote mapreduce job in Yarn with the 
configuration below, but I get the error |No FileSystem for scheme: 
hdfs|. The error is so also below. Why this is happening?

|          String host = "192.168.56.100";

            Configuration conf = job.getConfiguration();
            // this should be like defined in your yarn-site.xml
            conf.set("yarn.resourcemanager.address", host + ":50001");

             // framework is now "yarn", should be defined like this in mapred-site.xm
             conf.set("mapreduce.framework.name", "yarn");

             // like defined in hdfs-site.xml
             conf.set("fs.defaultFS", "hdfs://" + host + ":9000");

             for (Path inputPath : inputPaths)
                 FileInputFormat.addInputPath(job, new Path(conf.get("fs.defaultFS") + "/" + inputPath.toString()));

             FileOutputFormat.setOutputPath(job, new Path(conf.get("fs.defaultFS") + "/" + tempPath));

             job.waitForCompletion(true);
|

|Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
     at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
     at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
     at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
     at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
     at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:518)
     at org.apache.hadoop.mapred.examples.JobExecution.submit(JobExecution.java:169)
     at org.apache.hadoop.mapred.examples.WordCount.main(WordCount.java:101)
|

Thanks,

​

--------------000709030004030000060703
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=utf-8">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div style="" class="markdown-here-wrapper"
      data-md-url="Thunderbird">
      <p style="margin: 1.2em 0px ! important;">Hi,</p>
      <p style="margin: 1.2em 0px ! important;">I am trying to submit a
        remote mapreduce job in Yarn with the configuration below, but I
        get the error <code style="font-size: 0.85em; font-family:
          Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em;
          padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid
          rgb(234, 234, 234); background-color: rgb(248, 248, 248);
          border-radius: 3px; display: inline;">No FileSystem for
          scheme: hdfs</code>. The error is so also below. Why this is
        happening?</p>
      <pre style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;font-size: 1em; line-height: 1.2em;margin: 1.2em 0px;"><code style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em; padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid rgb(234, 234, 234); background-color: rgb(248, 248, 248); border-radius: 3px; display: inline;white-space: pre; overflow: auto; border-radius: 3px; border: 1px solid rgb(204, 204, 204); padding: 0.5em 0.7em; display: block ! important;">         String host = "192.168.56.100";

           Configuration conf = job.getConfiguration();
           // this should be like defined in your yarn-site.xml
           conf.set("yarn.resourcemanager.address", host + ":50001"); 

            // framework is now "yarn", should be defined like this in mapred-site.xm
            conf.set("mapreduce.framework.name", "yarn");

            // like defined in hdfs-site.xml
            conf.set("fs.defaultFS", "hdfs://" + host + ":9000");

            for (Path inputPath : inputPaths)
                FileInputFormat.addInputPath(job, new Path(conf.get("fs.defaultFS") + "/" + inputPath.toString()));

            FileOutputFormat.setOutputPath(job, new Path(conf.get("fs.defaultFS") + "/" + tempPath));

            job.waitForCompletion(true);
</code></pre>
      <pre style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;font-size: 1em; line-height: 1.2em;margin: 1.2em 0px;"><code style="font-size: 0.85em; font-family: Consolas,Inconsolata,Courier,monospace;margin: 0px 0.15em; padding: 0px 0.3em; white-space: pre-wrap; border: 1px solid rgb(234, 234, 234); background-color: rgb(248, 248, 248); border-radius: 3px; display: inline;white-space: pre; overflow: auto; border-radius: 3px; border: 1px solid rgb(204, 204, 204); padding: 0.5em 0.7em; display: block ! important;">Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:518)
    at org.apache.hadoop.mapred.examples.JobExecution.submit(JobExecution.java:169)
    at org.apache.hadoop.mapred.examples.WordCount.main(WordCount.java:101)
</code></pre>
      <p style="margin: 1.2em 0px ! important;">Thanks,</p>
      <div
title="MDH:SGksPGJyPjxicj5JIGFtIHRyeWluZyB0byBzdWJtaXQgYSByZW1vdGUgbWFwcmVkdWNlIGpvYiBpbiBZYXJuIHdpdGggdGhlIGNvbmZpZ3VyYXRpb24gYmVsb3csIGJ1dCBJIGdldCB0aGUgZXJyb3Ig
YE5vIEZpbGVTeXN0ZW0gZm9yIHNjaGVtZTogaGRmc2AuIFRoZSBlcnJvciBpcyBzbyBhbHNvIGJl
bG93LiBXaHkgdGhpcyBpcyBoYXBwZW5pbmc/PGJyPjxicj5gYGA8YnI+wqAgwqAgwqAgwqDCoCBT
dHJpbmcgaG9zdCA9ICIxOTIuMTY4LjU2LjEwMCI7PGJyPjxicj7CoCDCoCDCoCDCoCDCoMKgIENv
bmZpZ3VyYXRpb24gY29uZiA9IGpvYi5nZXRDb25maWd1cmF0aW9uKCk7PGJyPsKgwqDCoMKgwqDC
oMKgwqDCoMKgIC8vIHRoaXMgc2hvdWxkIGJlIGxpa2UgZGVmaW5lZCBpbiB5b3VyIHlhcm4tc2l0
ZS54bWw8YnI+wqAgwqAgwqDCoCDCoMKgwqAgY29uZi5zZXQoInlhcm4ucmVzb3VyY2VtYW5hZ2Vy
LmFkZHJlc3MiLCBob3N0ICsgIjo1MDAwMSIpOyA8YnI+PGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKg
IC8vIGZyYW1ld29yayBpcyBub3cgInlhcm4iLCBzaG91bGQgYmUgZGVmaW5lZCBsaWtlIHRoaXMg
aW4gbWFwcmVkLXNpdGUueG08YnI+wqDCoMKgIMKgwqDCoCDCoMKgwqAgY29uZi5zZXQoIm1hcHJl
ZHVjZS5mcmFtZXdvcmsubmFtZSIsICJ5YXJuIik7PGJyPjxicj7CoMKgwqAgwqDCoMKgIMKgwqDC
oCAvLyBsaWtlIGRlZmluZWQgaW4gaGRmcy1zaXRlLnhtbDxicj7CoMKgwqAgwqDCoMKgIMKgwqDC
oCBjb25mLnNldCgiZnMuZGVmYXVsdEZTIiwgImhkZnM6Ly8iICsgaG9zdCArICI6OTAwMCIpOzxi
cj48YnI+wqDCoMKgIMKgwqDCoCDCoMKgwqAgZm9yIChQYXRoIGlucHV0UGF0aCA6IGlucHV0UGF0
aHMpPGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKgIMKgwqDCoCBGaWxlSW5wdXRGb3JtYXQuYWRkSW5w
dXRQYXRoKGpvYiwgbmV3IFBhdGgoY29uZi5nZXQoImZzLmRlZmF1bHRGUyIpICsgIi8iICsgaW5w
dXRQYXRoLnRvU3RyaW5nKCkpKTs8YnI+PGJyPsKgwqDCoCDCoMKgwqAgwqDCoMKgIEZpbGVPdXRw
dXRGb3JtYXQuc2V0T3V0cHV0UGF0aChqb2IsIG5ldyBQYXRoKGNvbmYuZ2V0KCJmcy5kZWZhdWx0
RlMiKSArICIvIiArIHRlbXBQYXRoKSk7PGJyPjxicj7CoMKgwqAgwqDCoMKgIMKgwqDCoCBqb2Iu
d2FpdEZvckNvbXBsZXRpb24odHJ1ZSk7PGJyPjxicj5gYGA8YnI+PGJyPjxicj5gYGA8YnI+RXhj
ZXB0aW9uIGluIHRocmVhZCAibWFpbiIgamF2YS5pby5JT0V4Y2VwdGlvbjogTm8gRmlsZVN5c3Rl
bSBmb3Igc2NoZW1lOiBoZGZzPGJyPsKgwqDCoCBhdCBvcmcuYXBhY2hlLmhhZG9vcC5mcy5GaWxl
U3lzdGVtLmdldEZpbGVTeXN0ZW1DbGFzcyhGaWxlU3lzdGVtLmphdmE6MjU4NCk8YnI+wqDCoMKg
IGF0IG9yZy5hcGFjaGUuaGFkb29wLmZzLkZpbGVTeXN0ZW0uY3JlYXRlRmlsZVN5c3RlbShGaWxl
U3lzdGVtLmphdmE6MjU5MSk8YnI+wqDCoMKgIGF0IG9yZy5hcGFjaGUuaGFkb29wLmZzLkZpbGVT
eXN0ZW0uYWNjZXNzJDIwMChGaWxlU3lzdGVtLmphdmE6OTEpPGJyPsKgwqDCoCBhdCBvcmcuYXBh
Y2hlLmhhZG9vcC5mcy5GaWxlU3lzdGVtJENhY2hlLmdldEludGVybmFsKEZpbGVTeXN0ZW0uamF2
YToyNjMwKTxicj7CoMKgwqAgYXQgb3JnLmFwYWNoZS5oYWRvb3AuZnMuRmlsZVN5c3RlbSRDYWNo
ZS5nZXQoRmlsZVN5c3RlbS5qYXZhOjI2MTIpPGJyPsKgwqDCoCBhdCBvcmcuYXBhY2hlLmhhZG9v
cC5mcy5GaWxlU3lzdGVtLmdldChGaWxlU3lzdGVtLmphdmE6MzcwKTxicj7CoMKgwqAgYXQgb3Jn
LmFwYWNoZS5oYWRvb3AuZnMuUGF0aC5nZXRGaWxlU3lzdGVtKFBhdGguamF2YToyOTYpPGJyPsKg
wqDCoCBhdCBvcmcuYXBhY2hlLmhhZG9vcC5tYXByZWR1Y2UubGliLmlucHV0LkZpbGVJbnB1dEZv
cm1hdC5hZGRJbnB1dFBhdGgoRmlsZUlucHV0Rm9ybWF0LmphdmE6NTE4KTxicj7CoMKgwqAgYXQg
b3JnLmFwYWNoZS5oYWRvb3AubWFwcmVkLmV4YW1wbGVzLkpvYkV4ZWN1dGlvbi5zdWJtaXQoSm9i
RXhlY3V0aW9uLmphdmE6MTY5KTxicj7CoMKgwqAgYXQgb3JnLmFwYWNoZS5oYWRvb3AubWFwcmVk
LmV4YW1wbGVzLldvcmRDb3VudC5tYWluKFdvcmRDb3VudC5qYXZhOjEwMSk8YnI+YGBgPGJyPjxi
        cj5UaGFua3MsPGJyPjxicj48YnI+"
style="height:0;width:0;max-height:0;max-width:0;overflow:hidden;font-size:0em;padding:0;margin:0;">​</div>
    </div>
  </body>
</html>

--------------000709030004030000060703--