hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Boudot <abou...@cyres.fr>
Subject HDFS Federation Cluster IDs not matched
Date Wed, 25 Jul 2012 12:01:56 GMT
hello I'm beginner on technology hadoop and I encounter a problem ,
I am testing to  set 2 namenode of 2 separate host with HDFS federation and a pool block 
is not to create to startup namenode
log error here is the datanode

2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory
/couloir1/tmp/dfs/data is not formatted.
2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled
2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory
/couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681 is not formatted.
2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting block
pool BP-459972778-172.31.6.11-1343208016681 directory /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681/current
2012-07-25 11:20:39,860 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage:
nsid=91302135;bpid=BP-459972778-172.31.6.11-1343208016681;lv=-40;nsInfo=lv=-40;cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8;nsid=91302135;c=0;bpid=BP-459972778-172.31.6.11-1343208016681
2012-07-25 11:20:39,870 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:
Added volume - /couloir1/tmp/dfs/data/current
2012-07-25 11:20:39,876 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:
Registered FSDatasetState MBean
2012-07-25 11:20:39,880 INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic
Directory Tree Verification scan starting at 1343224800880 with interval 21600000
2012-07-25 11:20:39,881 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl:
Adding block pool BP-459972778-172.31.6.11-1343208016681
2012-07-25 11:20:39,894 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-459972778-172.31.6.11-1343208016681
(storage id DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-n10/172.31.6.11:33333
beginning handshake with NN
2012-07-25 11:20:39,899 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization
failed for block pool Block pool BP-321278605-172.31.4.59-1343207919699 (storage id DS-1163140142-172.31.6.10-50010-1343208039823)
service to hadoop-vm03/172.31.4.59:33333
java.io.IOException: Cluster IDs not matched: dn cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8
but ns cid=CID-9f7751ec-615b-4f3b-8644-5583ba263a0e; bpid=BP-321278605-172.31.4.59-1343207919699
        at org.apache.hadoop.hdfs.server.datanode.DataNode.setClusterId(DataNode.java:307)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:801)
        at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:308)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:217)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660)
        at java.lang.Thread.run(Thread.java:662)
2012-07-25 11:20:39,899 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block
pool service for: Block pool BP-321278605-172.31.4.59-1343207919699 (storage id DS-1163140142-172.31.6.10-50010-1343208039823)
service to hadoop-vm03/172.31.4.59:33333
2012-07-25 11:20:39,900 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block
pool BP-321278605-172.31.4.59-1343207919699 (storage id DS-1163140142-172.31.6.10-50010-1343208039823)

Here config  for core-site.xml

<property>
      <name>hadoop.tmp.dir</name>
              <value>/couloir1/tmp</value>
                         </property>

Here config  for hdfs-site.xml

<property>
    <name>dfs.federation.nameservices</name>
              <value>ns1,ns2</value>
</property>
<property>
<name>dfs.permissions</name>
  <value>false</value>
  </property>
<property>
<name>dfs.replication</name>
<value>2</value>
    </property>
<property>
<name>dfs.namenode.rpc-address.ns1</name>
<value>hadoop-vm03:33333</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns2</name>
  <value>hadoop-n10:33333</value>
    </property>
</configuration>

Help me please
Thank you



Mime
View raw message