hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From johny casanova <pcgamer2...@outlook.com>
Subject RE: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"
Date Mon, 15 Dec 2014 18:24:02 GMT
do you have selinux and iptables turned off?
 



Date: Mon, 15 Dec 2014 09:54:41 -0600
Subject: Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"
From: dongdan39@gmail.com
To: user@hadoop.apache.org






Found in the log file:
2014-12-12 15:51:10,434 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.IllegalArgumentException:
Does not contain a valid host:port authority: file:///
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:212)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:244)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:280)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)

But I have set it in core-site.xml already:
<property>  
  <name>fs.defaultFS</name>  
  <value>hdfs://master-node:9000</value>  
</property>

Other settings:
$ cat mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>master-node:9002</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master-node:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master-node:19888</value>
</property>
</configuration>


$ cat yarn-site.xml
<configuration>

<!-- Site specific YARN configuration properties -->
<property>  
   <name>mapreduce.framework.name</name>  
   <value>yarn</value>  
</property>  
<property>  
   <name>yarn.resourcemanager.address</name>  
   <value>master-node:18040</value>  
</property>  
<property>  
   <name>yarn.resourcemanager.scheduler.address</name>  
   <value>master-node:18030</value>  
</property>  
<property>  
   <name>yarn.resourcemanager.webapp.address</name>  
   <value>master-node:18088</value>  
</property>  
<property>  
   <name>yarn.resourcemanager.resource-tracker.address</name>  
   <value>master-node:18025</value>  
</property>  
<property>  
   <name>yarn.resourcemanager.admin.address</name>  
   <value>master-node:18141</value>  
</property>  
<property>  
   <name>yarn.nodemanager.aux-services</name>  
   <value>mapreduce_shuffle</value>  
</property>  
<property>  
   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>  
   <value>org.apache.hadoop.mapred.ShuffleHandler</value>  
</property>  
</configuration>

Cheers,
Dan




2014-12-15 9:17 GMT-06:00 Dan Dong <dongdan39@gmail.com>:

Thank you all, but still the same after change file:/ to file://, and HADOOP_CONF_DIR points
to the correct position already:
$ echo $HADOOP_CONF_DIR 
/home/dong/import/hadoop-2.6.0/etc/hadoop




2014-12-15 8:57 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:




Don't you have to use file:// instead of just one /?
 



From: brahmareddy.battula@huawei.com
To: user@hadoop.apache.org
Subject: RE: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"
Date: Sat, 13 Dec 2014 05:48:18 +0000




Hi Dong,

HADOOP_CONF_DIR might be referring to default..you can export HADOOP_CONF_DIR where following
configuration files are present..



Thanks & Regards
Brahma Reddy Battula






From: Dan Dong [dongdan39@gmail.com]
Sent: Saturday, December 13, 2014 3:43 AM
To: user@hadoop.apache.org
Subject: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"










Hi, 
  I installed Hadoop2.6.0 on my cluster with 2 nodes, I got the following error when I run:
$hadoop dfsadmin -report
FileSystem file:/// is not a distributed file system

What this mean? I have set it in core-site.xml already:
<property>  
  <name>fs.defaultFS</name>  
  <value>hdfs://master-node:9000</value>  
</property> 

and in hdfs-site.xml:
<property>   
  <name>dfs.namenode.name.dir</name>   
  <value>file:/home/dong/hadoop-2.6.0-dist/dfs/name</value>   
  <final>true</final>  
</property>   
<property>   
  <name>dfs.dataname.data.dir</name>   
  <value>file:/home/dong/hadoop-2.6.0-dist/dfs/data</value>   
  <final>true</final>  
</property>  

The java process are running on master as:
10479 SecondaryNameNode
10281 NameNode
10628 ResourceManager

and on slave:
22870 DataNode
22991 NodeManager

Any hints? Thanks!

Cheers,
Dan




 		 	   		  
Mime
View raw message