hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiayu Ji <jiayu...@gmail.com>
Subject Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"
Date Tue, 16 Dec 2014 20:39:32 GMT
The cluster is running hadoop 2.0 while you client side in under hadoop
1.0.

I would guess you have installed 1.0 on your client machine before and your
env variable is still pointing to it.

On Tue, Dec 16, 2014 at 9:31 AM, Dan Dong <dongdan39@gmail.com> wrote:
>
> Thanks, the error now changes to the following:
> $ hadoop dfsadmin -report
> report: Server IPC version 9 cannot communicate with client version 4
>
> Not clear which Server and which client are conflicting. All hadoop
> components comes from the hadoop-2.6.0.tar.gz package, what's going wrong?
>
> Cheers,
> Dan
>
>
> 2014-12-15 22:30 GMT-06:00 Susheel Kumar Gadalay <skgadalay@gmail.com>:
>
>> Give complete hostname with domain name not just master-node.
>>
>> <property>
>>   <name>fs.defaultFS</name>
>>   <value>hdfs://master-node.domain.name:9000</value>
>> </property>
>>
>> Else give IP address also
>>
>>
>> On 12/16/14, Dan Dong <dongdan39@gmail.com> wrote:
>> > Hi, Johny,
>> >   Yes, they have been turned off from the beginning. Guess the problem
>> is
>> > still in the conf files, it would be helpful if some example *.xml
>> could be
>> > shown.
>> >
>> >   Cheers,
>> >   Dan
>> >
>> >
>> > 2014-12-15 12:24 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:
>> >>
>> >> do you have selinux and iptables turned off?
>> >>
>> >>  ------------------------------
>> >> Date: Mon, 15 Dec 2014 09:54:41 -0600
>> >> Subject: Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed
>> file
>> >> system"
>> >> From: dongdan39@gmail.com
>> >> To: user@hadoop.apache.org
>> >>
>> >>
>> >>   Found in the log file:
>> >> 2014-12-12 15:51:10,434 ERROR
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode:
>> >> java.lang.IllegalArgumentException: Does not contain a valid host:port
>> >> authority: file:///
>> >>         at
>> >> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
>> >>         at
>> >>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:212)
>> >>         at
>> >>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:244)
>> >>         at
>> >>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:280)
>> >>         at
>> >>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569)
>> >>         at
>> >>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479)
>> >>         at
>> >>
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)
>> >>
>> >> But I have set it in core-site.xml already:
>> >> <property>
>> >>   <name>fs.defaultFS</name>
>> >>   <value>hdfs://master-node:9000</value>
>> >> </property>
>> >>
>> >> Other settings:
>> >> $ cat mapred-site.xml
>> >> <configuration>
>> >> <property>
>> >> <name>mapred.job.tracker</name>
>> >> <value>master-node:9002</value>
>> >> </property>
>> >> <property>
>> >> <name>mapreduce.jobhistory.address</name>
>> >> <value>master-node:10020</value>
>> >> </property>
>> >> <property>
>> >> <name>mapreduce.jobhistory.webapp.address</name>
>> >> <value>master-node:19888</value>
>> >> </property>
>> >> </configuration>
>> >>
>> >> $ cat yarn-site.xml
>> >> <configuration>
>> >>
>> >> <!-- Site specific YARN configuration properties -->
>> >> <property>
>> >>    <name>mapreduce.framework.name</name>
>> >>    <value>yarn</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.resourcemanager.address</name>
>> >>    <value>master-node:18040</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.resourcemanager.scheduler.address</name>
>> >>    <value>master-node:18030</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.resourcemanager.webapp.address</name>
>> >>    <value>master-node:18088</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.resourcemanager.resource-tracker.address</name>
>> >>    <value>master-node:18025</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.resourcemanager.admin.address</name>
>> >>    <value>master-node:18141</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.nodemanager.aux-services</name>
>> >>    <value>mapreduce_shuffle</value>
>> >> </property>
>> >> <property>
>> >>    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
>> >>    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>> >> </property>
>> >> </configuration>
>> >>
>> >> Cheers,
>> >> Dan
>> >>
>> >>
>> >> 2014-12-15 9:17 GMT-06:00 Dan Dong <dongdan39@gmail.com>:
>> >>
>> >> Thank you all, but still the same after change file:/ to file://, and
>> >> HADOOP_CONF_DIR points to the correct position already:
>> >> $ echo $HADOOP_CONF_DIR
>> >> /home/dong/import/hadoop-2.6.0/etc/hadoop
>> >>
>> >>
>> >> 2014-12-15 8:57 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:
>> >>
>> >>  Don't you have to use file:// instead of just one /?
>> >>
>> >>  ------------------------------
>> >> From: brahmareddy.battula@huawei.com
>> >> To: user@hadoop.apache.org
>> >> Subject: RE: Hadoop 2.6.0: "FileSystem file:/// is not a distributed
>> file
>> >> system"
>> >> Date: Sat, 13 Dec 2014 05:48:18 +0000
>> >>
>> >>
>> >> Hi Dong,
>> >>
>> >> HADOOP_CONF_DIR might be referring to default..you can export
>> >> HADOOP_CONF_DIR where following configuration files are present..
>> >>
>> >> Thanks & Regards
>> >> Brahma Reddy Battula
>> >>
>> >>
>> >>  ------------------------------
>> >> *From:* Dan Dong [dongdan39@gmail.com]
>> >> *Sent:* Saturday, December 13, 2014 3:43 AM
>> >> *To:* user@hadoop.apache.org
>> >> *Subject:* Hadoop 2.6.0: "FileSystem file:/// is not a distributed file
>> >> system"
>> >>
>> >>     Hi,
>> >>   I installed Hadoop2.6.0 on my cluster with 2 nodes, I got the
>> following
>> >> error when I run:
>> >> $hadoop dfsadmin -report
>> >> FileSystem file:/// is not a distributed file system
>> >>
>> >> What this mean? I have set it in core-site.xml already:
>> >> <property>
>> >>   <name>fs.defaultFS</name>
>> >>   <value>hdfs://master-node:9000</value>
>> >> </property>
>> >>
>> >> and in hdfs-site.xml:
>> >> <property>
>> >>   <name>dfs.namenode.name.dir</name>
>> >>   <value>file:/home/dong/hadoop-2.6.0-dist/dfs/name</value>
>> >>   <final>true</final>
>> >> </property>
>> >> <property>
>> >>   <name>dfs.dataname.data.dir</name>
>> >>   <value>file:/home/dong/hadoop-2.6.0-dist/dfs/data</value>
>> >>   <final>true</final>
>> >> </property>
>> >>
>> >> The java process are running on master as:
>> >> 10479 SecondaryNameNode
>> >> 10281 NameNode
>> >> 10628 ResourceManager
>> >>
>> >> and on slave:
>> >> 22870 DataNode
>> >> 22991 NodeManager
>> >>
>> >> Any hints? Thanks!
>> >>
>> >> Cheers,
>> >> Dan
>> >>
>> >>
>> >>
>> >
>>
>

-- 
Regards,

Jiayu (James) Ji,

Cell: (312)823-7393

Mime
View raw message