hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elad Itzhakian <el...@mellanox.com>
Subject RE: Error while configuring HDFS fedration
Date Mon, 23 Sep 2013 14:05:47 GMT
Ports in use may result from actual processes using them, or just ghost processes. The second
error may be caused by inconsistent permissions on different nodes, and/or a format is needed
on DFS.

I suggest the following:


1.       sbin/stop-dfs.sh && sbin/stop-yarn.sh

2.       sudo killall java (on all nodes)

3.       sudo chmod -R 755 /home/lab/hadoop-2.1.0-beta/tmp/dfs (on all nodes)

4.       sudo rm -rf /home/lab/hadoop-2.1.0-beta/tmp/dfs/* (on all nodes)

5.       bin/hdfs namenode -format -force

6.       sbin/start-dfs.sh && sbin/start-yarn.sh

Then see if you get that error again.

From: Manickam P [mailto:manickam.p@outlook.com]
Sent: Monday, September 23, 2013 4:44 PM
To: user@hadoop.apache.org
Subject: Error while configuring HDFS fedration

Guys,

I'm trying to configure HDFS federation with 2.1.0 beta version. I am having 3 machines in
that i want to have two name nodes and one data node.

I have done the other thing like password less ssh and host entries properly. when i start
the cluster i'm getting the below error.

In node one i'm getting this error.
java.net.BindException: Port in use: lab-hadoop.eng.com:50070

In another node i'm getting this error.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /home/lab/hadoop-2.1.0-beta/tmp/dfs/name
is in an inconsistent state: storage directory does not exist or is not accessible.

My core-site xml has the below.
<configuration>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://10.101.89.68:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/lab/hadoop-2.1.0-beta/tmp</value>
  </property>
</configuration>

My hdfs-site xml has the below.
<configuration>
   <property>
     <name>dfs.replication</name>
     <value>2</value>
   </property>
   <property>
     <name>dfs.permissions</name>
     <value>false</value>
   </property>
   <property>
        <name>dfs.federation.nameservices</name>
        <value>ns1,ns2</value>
    </property>
    <property>
        <name>dfs.namenode.rpc-address.ns1</name>
        <value>10.101.89.68:9001</value>
    </property>
   <property>
    <name>dfs.namenode.http-address.ns1</name>
    <value>10.101.89.68:50070</value>
   </property>
   <property>
        <name>dfs.namenode.secondary.http-address.ns1</name>
        <value>10.101.89.68:50090</value>
    </property>
    <property>
        <name>dfs.namenode.rpc-address.ns2</name>
        <value>10.101.89.69:9001</value>
    </property>
   <property>
    <name>dfs.namenode.http-address.ns2</name>
    <value>10.101.89.69:50070</value>
   </property>
   <property>
        <name>dfs.namenode.secondary.http-address.ns2</name>
        <value>10.101.89.69:50090</value>
    </property>
 </configuration>

Please help me to fix this error.


Thanks,
Manickam P


Mime
View raw message