hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Dong <dongda...@gmail.com>
Subject Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"
Date Mon, 15 Dec 2014 20:00:26 GMT
Hi, Johny,
  Yes, they have been turned off from the beginning. Guess the problem is
still in the conf files, it would be helpful if some example *.xml could be
shown.

  Cheers,
  Dan


2014-12-15 12:24 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:
>
> do you have selinux and iptables turned off?
>
>  ------------------------------
> Date: Mon, 15 Dec 2014 09:54:41 -0600
> Subject: Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file
> system"
> From: dongdan39@gmail.com
> To: user@hadoop.apache.org
>
>
>   Found in the log file:
> 2014-12-12 15:51:10,434 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode:
> java.lang.IllegalArgumentException: Does not contain a valid host:port
> authority: file:///
>         at
> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:212)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:244)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:280)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)
>
> But I have set it in core-site.xml already:
> <property>
>   <name>fs.defaultFS</name>
>   <value>hdfs://master-node:9000</value>
> </property>
>
> Other settings:
> $ cat mapred-site.xml
> <configuration>
> <property>
> <name>mapred.job.tracker</name>
> <value>master-node:9002</value>
> </property>
> <property>
> <name>mapreduce.jobhistory.address</name>
> <value>master-node:10020</value>
> </property>
> <property>
> <name>mapreduce.jobhistory.webapp.address</name>
> <value>master-node:19888</value>
> </property>
> </configuration>
>
> $ cat yarn-site.xml
> <configuration>
>
> <!-- Site specific YARN configuration properties -->
> <property>
>    <name>mapreduce.framework.name</name>
>    <value>yarn</value>
> </property>
> <property>
>    <name>yarn.resourcemanager.address</name>
>    <value>master-node:18040</value>
> </property>
> <property>
>    <name>yarn.resourcemanager.scheduler.address</name>
>    <value>master-node:18030</value>
> </property>
> <property>
>    <name>yarn.resourcemanager.webapp.address</name>
>    <value>master-node:18088</value>
> </property>
> <property>
>    <name>yarn.resourcemanager.resource-tracker.address</name>
>    <value>master-node:18025</value>
> </property>
> <property>
>    <name>yarn.resourcemanager.admin.address</name>
>    <value>master-node:18141</value>
> </property>
> <property>
>    <name>yarn.nodemanager.aux-services</name>
>    <value>mapreduce_shuffle</value>
> </property>
> <property>
>    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
>    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
> </property>
> </configuration>
>
> Cheers,
> Dan
>
>
> 2014-12-15 9:17 GMT-06:00 Dan Dong <dongdan39@gmail.com>:
>
> Thank you all, but still the same after change file:/ to file://, and
> HADOOP_CONF_DIR points to the correct position already:
> $ echo $HADOOP_CONF_DIR
> /home/dong/import/hadoop-2.6.0/etc/hadoop
>
>
> 2014-12-15 8:57 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:
>
>  Don't you have to use file:// instead of just one /?
>
>  ------------------------------
> From: brahmareddy.battula@huawei.com
> To: user@hadoop.apache.org
> Subject: RE: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file
> system"
> Date: Sat, 13 Dec 2014 05:48:18 +0000
>
>
> Hi Dong,
>
> HADOOP_CONF_DIR might be referring to default..you can export
> HADOOP_CONF_DIR where following configuration files are present..
>
> Thanks & Regards
> Brahma Reddy Battula
>
>
>  ------------------------------
> *From:* Dan Dong [dongdan39@gmail.com]
> *Sent:* Saturday, December 13, 2014 3:43 AM
> *To:* user@hadoop.apache.org
> *Subject:* Hadoop 2.6.0: "FileSystem file:/// is not a distributed file
> system"
>
>     Hi,
>   I installed Hadoop2.6.0 on my cluster with 2 nodes, I got the following
> error when I run:
> $hadoop dfsadmin -report
> FileSystem file:/// is not a distributed file system
>
> What this mean? I have set it in core-site.xml already:
> <property>
>   <name>fs.defaultFS</name>
>   <value>hdfs://master-node:9000</value>
> </property>
>
> and in hdfs-site.xml:
> <property>
>   <name>dfs.namenode.name.dir</name>
>   <value>file:/home/dong/hadoop-2.6.0-dist/dfs/name</value>
>   <final>true</final>
> </property>
> <property>
>   <name>dfs.dataname.data.dir</name>
>   <value>file:/home/dong/hadoop-2.6.0-dist/dfs/data</value>
>   <final>true</final>
> </property>
>
> The java process are running on master as:
> 10479 SecondaryNameNode
> 10281 NameNode
> 10628 ResourceManager
>
> and on slave:
> 22870 DataNode
> 22991 NodeManager
>
> Any hints? Thanks!
>
> Cheers,
> Dan
>
>
>

Mime
View raw message