hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Horrocks (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-6952) Update Rack Aware documentation and/or script
Date Wed, 27 Aug 2014 10:29:57 GMT
Chris Horrocks created HDFS-6952:
------------------------------------

             Summary: Update Rack Aware documentation and/or script
                 Key: HDFS-6952
                 URL: https://issues.apache.org/jira/browse/HDFS-6952
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: balancer
    Affects Versions: 2.5.0
            Reporter: Chris Horrocks
            Priority: Minor


The Rack Aware documentation references a rack-topology.sh script which has two small flaws;

1) From 2.x.x the default config dir is ..etc/hadoop not ..etc/hadoop/conf
2) When configuring DN to rack IDs in the rack_topology.data file if hostnames are used then
the rack-topology.sh script returns the prefixed rack ID but the balancer and fsck report
omit the rack ID and only return one single rack (IP addresses in the data file work fine).

(e.g: when using hostnames:
rack-topology.sh
-----------------------
RACK_PREFIX=LAB
..
HADOOP_CONF=${HADOOP_CONF:-"/usr/local/hadoop/hadoop-2.5.0/etc/hadoop"}

rack_topology.data
------------------------
datanode0 01
..

grep NetworkTopology logs/hadoop-hduser-namenode-NameNode0.log 
--------------------------------------------------------------------------------------------
2014-08-27 10:29:52,518 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /LAB/*rack*/192.168.0.12:50010

hdfs fsck /
-------------
 Number of data-nodes:          3
 Number of racks:               1)



(e.g. when using IP addresses:
rack-topology.sh
-----------------------
RACK_PREFIX=LAB
..
HADOOP_CONF=${HADOOP_CONF:-"/usr/local/hadoop/hadoop-2.5.0/etc/hadoop"}

rack_topology.data
-----------------------
192.168.0.10 01
..
grep NetworkTopology logs/hadoop-hduser-namenode-NameNode0.log 
-----------------------
2014-08-27 11:14:22,796 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /LAB/*rack_01*/192.168.0.10:50010

hdfs fsck /
-------------
 Number of data-nodes:          3
 Number of racks:               2)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message