hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Re: HDFS Balancer
Date Tue, 07 Mar 2017 14:37:20 GMT
Hey Ted,

Thanks Cpt. Obvious :)

I know how to use "blame" or git log how to find the JIRA, but what I was after is how that
- apparently wrong - information came about. And if it is wrong, what _is_ the current status
of this feature. 

I do believe this is an important operational piece as it helps with rearranging clusters.
Since it seems to still be missing, I am wondering what needs to be done here. 

Makes sense?

Lars

Sent from my iPhone

> On 6 Mar 2017, at 19:50, Ted Yu <yuzhihong@gmail.com> wrote:
> 
> w.r.t. the first question, the quoted paragraph came from:
> 
> HBASE-15332 Document how to take advantage of HDFS-6133 in HBase
> 
>> On Mon, Mar 6, 2017 at 6:38 PM, Lars George <lars.george@gmail.com> wrote:
>> 
>> Hi,
>> 
>> I am trying to grok what came out of all these issues about the HDFS
>> balancer and being able to avoid it destroying HBase locality. There
>> is this https://issues.apache.org/jira/browse/HBASE-13021 from JM, and
>> the book http://hbase.apache.org/book.html#_hbase_and_hdfs refers to
>> https://issues.apache.org/jira/browse/HDFS-6133, stating:
>> 
>> "HDFS-6133 provides the ability to exclude a given directory from the
>> HDFS load balancer, by setting the dfs.datanode.block-pinning.enabled
>> property to true in your HDFS configuration and running the following
>> hdfs command:
>> 
>> $ sudo -u hdfs hdfs balancer -exclude /hbase"
>> 
>> I checked the Balancer class in 2.7.2 and it does not have that
>> support, i.e. being able to exclude a path, it can only exclude hosts.
>> That is also clear from HDFS-6133, which adds favoured nodes, but not
>> being able to exclude paths (which would be nice).
>> 
>> HBASE-13021 mentions that this works in tandem with the HBase favored
>> node feature, but that makes it much more complicated since you have
>> to pin individual regions to nodes, instead of doing that wholesale.
>> 
>> Where does the above in the HBase book come from, and what is the
>> current state as far as you know?
>> 
>> Cheers,
>> Lars
>> 

Mime
View raw message