hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject HDFS Balancer
Date Tue, 07 Mar 2017 02:38:32 GMT

I am trying to grok what came out of all these issues about the HDFS
balancer and being able to avoid it destroying HBase locality. There
is this https://issues.apache.org/jira/browse/HBASE-13021 from JM, and
the book http://hbase.apache.org/book.html#_hbase_and_hdfs refers to
https://issues.apache.org/jira/browse/HDFS-6133, stating:

"HDFS-6133 provides the ability to exclude a given directory from the
HDFS load balancer, by setting the dfs.datanode.block-pinning.enabled
property to true in your HDFS configuration and running the following
hdfs command:

$ sudo -u hdfs hdfs balancer -exclude /hbase"

I checked the Balancer class in 2.7.2 and it does not have that
support, i.e. being able to exclude a path, it can only exclude hosts.
That is also clear from HDFS-6133, which adds favoured nodes, but not
being able to exclude paths (which would be nice).

HBASE-13021 mentions that this works in tandem with the HBase favored
node feature, but that makes it much more complicated since you have
to pin individual regions to nodes, instead of doing that wholesale.

Where does the above in the HBase book come from, and what is the
current state as far as you know?


View raw message