hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Dong <dongda...@gmail.com>
Subject Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed file system"
Date Tue, 16 Dec 2014 21:16:46 GMT
I installed the hadoop by untar the hadoop-2.6.0.tar.gz, will check
further. Thanks.

2014-12-16 14:39 GMT-06:00 Jiayu Ji <jiayu.ji@gmail.com>:
>
> The cluster is running hadoop 2.0 while you client side in under hadoop
> 1.0.
>
> I would guess you have installed 1.0 on your client machine before and
> your env variable is still pointing to it.
>
> On Tue, Dec 16, 2014 at 9:31 AM, Dan Dong <dongdan39@gmail.com> wrote:
>>
>> Thanks, the error now changes to the following:
>> $ hadoop dfsadmin -report
>> report: Server IPC version 9 cannot communicate with client version 4
>>
>> Not clear which Server and which client are conflicting. All hadoop
>> components comes from the hadoop-2.6.0.tar.gz package, what's going wrong?
>>
>> Cheers,
>> Dan
>>
>>
>> 2014-12-15 22:30 GMT-06:00 Susheel Kumar Gadalay <skgadalay@gmail.com>:
>>
>>> Give complete hostname with domain name not just master-node.
>>>
>>> <property>
>>>   <name>fs.defaultFS</name>
>>>   <value>hdfs://master-node.domain.name:9000</value>
>>> </property>
>>>
>>> Else give IP address also
>>>
>>>
>>> On 12/16/14, Dan Dong <dongdan39@gmail.com> wrote:
>>> > Hi, Johny,
>>> >   Yes, they have been turned off from the beginning. Guess the problem
>>> is
>>> > still in the conf files, it would be helpful if some example *.xml
>>> could be
>>> > shown.
>>> >
>>> >   Cheers,
>>> >   Dan
>>> >
>>> >
>>> > 2014-12-15 12:24 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:
>>> >>
>>> >> do you have selinux and iptables turned off?
>>> >>
>>> >>  ------------------------------
>>> >> Date: Mon, 15 Dec 2014 09:54:41 -0600
>>> >> Subject: Re: Hadoop 2.6.0: "FileSystem file:/// is not a distributed
>>> file
>>> >> system"
>>> >> From: dongdan39@gmail.com
>>> >> To: user@hadoop.apache.org
>>> >>
>>> >>
>>> >>   Found in the log file:
>>> >> 2014-12-12 15:51:10,434 ERROR
>>> >> org.apache.hadoop.hdfs.server.namenode.NameNode:
>>> >> java.lang.IllegalArgumentException: Does not contain a valid host:port
>>> >> authority: file:///
>>> >>         at
>>> >> org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
>>> >>         at
>>> >>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:212)
>>> >>         at
>>> >>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:244)
>>> >>         at
>>> >>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:280)
>>> >>         at
>>> >>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569)
>>> >>         at
>>> >>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479)
>>> >>         at
>>> >>
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)
>>> >>
>>> >> But I have set it in core-site.xml already:
>>> >> <property>
>>> >>   <name>fs.defaultFS</name>
>>> >>   <value>hdfs://master-node:9000</value>
>>> >> </property>
>>> >>
>>> >> Other settings:
>>> >> $ cat mapred-site.xml
>>> >> <configuration>
>>> >> <property>
>>> >> <name>mapred.job.tracker</name>
>>> >> <value>master-node:9002</value>
>>> >> </property>
>>> >> <property>
>>> >> <name>mapreduce.jobhistory.address</name>
>>> >> <value>master-node:10020</value>
>>> >> </property>
>>> >> <property>
>>> >> <name>mapreduce.jobhistory.webapp.address</name>
>>> >> <value>master-node:19888</value>
>>> >> </property>
>>> >> </configuration>
>>> >>
>>> >> $ cat yarn-site.xml
>>> >> <configuration>
>>> >>
>>> >> <!-- Site specific YARN configuration properties -->
>>> >> <property>
>>> >>    <name>mapreduce.framework.name</name>
>>> >>    <value>yarn</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.resourcemanager.address</name>
>>> >>    <value>master-node:18040</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.resourcemanager.scheduler.address</name>
>>> >>    <value>master-node:18030</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.resourcemanager.webapp.address</name>
>>> >>    <value>master-node:18088</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.resourcemanager.resource-tracker.address</name>
>>> >>    <value>master-node:18025</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.resourcemanager.admin.address</name>
>>> >>    <value>master-node:18141</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.nodemanager.aux-services</name>
>>> >>    <value>mapreduce_shuffle</value>
>>> >> </property>
>>> >> <property>
>>> >>    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
>>> >>    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>>> >> </property>
>>> >> </configuration>
>>> >>
>>> >> Cheers,
>>> >> Dan
>>> >>
>>> >>
>>> >> 2014-12-15 9:17 GMT-06:00 Dan Dong <dongdan39@gmail.com>:
>>> >>
>>> >> Thank you all, but still the same after change file:/ to file://, and
>>> >> HADOOP_CONF_DIR points to the correct position already:
>>> >> $ echo $HADOOP_CONF_DIR
>>> >> /home/dong/import/hadoop-2.6.0/etc/hadoop
>>> >>
>>> >>
>>> >> 2014-12-15 8:57 GMT-06:00 johny casanova <pcgamer2426@outlook.com>:
>>> >>
>>> >>  Don't you have to use file:// instead of just one /?
>>> >>
>>> >>  ------------------------------
>>> >> From: brahmareddy.battula@huawei.com
>>> >> To: user@hadoop.apache.org
>>> >> Subject: RE: Hadoop 2.6.0: "FileSystem file:/// is not a distributed
>>> file
>>> >> system"
>>> >> Date: Sat, 13 Dec 2014 05:48:18 +0000
>>> >>
>>> >>
>>> >> Hi Dong,
>>> >>
>>> >> HADOOP_CONF_DIR might be referring to default..you can export
>>> >> HADOOP_CONF_DIR where following configuration files are present..
>>> >>
>>> >> Thanks & Regards
>>> >> Brahma Reddy Battula
>>> >>
>>> >>
>>> >>  ------------------------------
>>> >> *From:* Dan Dong [dongdan39@gmail.com]
>>> >> *Sent:* Saturday, December 13, 2014 3:43 AM
>>> >> *To:* user@hadoop.apache.org
>>> >> *Subject:* Hadoop 2.6.0: "FileSystem file:/// is not a distributed
>>> file
>>> >> system"
>>> >>
>>> >>     Hi,
>>> >>   I installed Hadoop2.6.0 on my cluster with 2 nodes, I got the
>>> following
>>> >> error when I run:
>>> >> $hadoop dfsadmin -report
>>> >> FileSystem file:/// is not a distributed file system
>>> >>
>>> >> What this mean? I have set it in core-site.xml already:
>>> >> <property>
>>> >>   <name>fs.defaultFS</name>
>>> >>   <value>hdfs://master-node:9000</value>
>>> >> </property>
>>> >>
>>> >> and in hdfs-site.xml:
>>> >> <property>
>>> >>   <name>dfs.namenode.name.dir</name>
>>> >>   <value>file:/home/dong/hadoop-2.6.0-dist/dfs/name</value>
>>> >>   <final>true</final>
>>> >> </property>
>>> >> <property>
>>> >>   <name>dfs.dataname.data.dir</name>
>>> >>   <value>file:/home/dong/hadoop-2.6.0-dist/dfs/data</value>
>>> >>   <final>true</final>
>>> >> </property>
>>> >>
>>> >> The java process are running on master as:
>>> >> 10479 SecondaryNameNode
>>> >> 10281 NameNode
>>> >> 10628 ResourceManager
>>> >>
>>> >> and on slave:
>>> >> 22870 DataNode
>>> >> 22991 NodeManager
>>> >>
>>> >> Any hints? Thanks!
>>> >>
>>> >> Cheers,
>>> >> Dan
>>> >>
>>> >>
>>> >>
>>> >
>>>
>>
>
> --
> Regards,
>
> Jiayu (James) Ji,
>
> Cell: (312)823-7393
>
>

Mime
View raw message