hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Re: HDFS Balancer
Date Sat, 11 Mar 2017 05:25:10 GMT
Yeah, my thoughts exactly... though thanks Harsh for taking action to
clean up the documentation! Good on you.

On Thu, Mar 9, 2017 at 11:01 AM, Jean-Marc Spaggiari
<jean-marc@spaggiari.org> wrote:
> So there is no way to use the pinning feature without having to use the
> favored nodes option? :(
>
> Le 2017-03-08 6:13 AM, "Harsh J" <harsh@cloudera.com> a écrit :
>
>> Hey Lars!,
>>
>> I was on a similar line of investigation today, and I've filed
>> https://issues.apache.org/jira/browse/HBASE-17760 to change the text. The
>> pinning part of the text is relevant, but the command part isn't. In
>> addition, you'd need to manually use the FavoredNodeLoadBalancer work to
>> actually get HBase to apply pinning to its writes by passing around proper
>> favored-node hint hostnames. I've also linked past and future relevant work
>> JIRAs to that one.
>>
>> Stumbled on this email when searching some follow-throughs, thought I'd
>> drop a note.
>>
>> On Tue, 7 Mar 2017 at 20:18 Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> > bq. how that - apparently wrong - information came about
>> >
>> > Maybe Sean / Misty can give some context.
>> >
>> > Cheers
>> >
>> > On Tue, Mar 7, 2017 at 6:37 AM, Lars George <lars.george@gmail.com>
>> wrote:
>> >
>> > > Hey Ted,
>> > >
>> > > Thanks Cpt. Obvious :)
>> > >
>> > > I know how to use "blame" or git log how to find the JIRA, but what I
>> was
>> > > after is how that - apparently wrong - information came about. And if
>> it
>> > is
>> > > wrong, what _is_ the current status of this feature.
>> > >
>> > > I do believe this is an important operational piece as it helps with
>> > > rearranging clusters. Since it seems to still be missing, I am
>> wondering
>> > > what needs to be done here.
>> > >
>> > > Makes sense?
>> > >
>> > > Lars
>> > >
>> > > Sent from my iPhone
>> > >
>> > > > On 6 Mar 2017, at 19:50, Ted Yu <yuzhihong@gmail.com> wrote:
>> > > >
>> > > > w.r.t. the first question, the quoted paragraph came from:
>> > > >
>> > > > HBASE-15332 Document how to take advantage of HDFS-6133 in HBase
>> > > >
>> > > >> On Mon, Mar 6, 2017 at 6:38 PM, Lars George <lars.george@gmail.com>
>> > > wrote:
>> > > >>
>> > > >> Hi,
>> > > >>
>> > > >> I am trying to grok what came out of all these issues about the
HDFS
>> > > >> balancer and being able to avoid it destroying HBase locality.
There
>> > > >> is this https://issues.apache.org/jira/browse/HBASE-13021 from
JM,
>> > and
>> > > >> the book http://hbase.apache.org/book.html#_hbase_and_hdfs refers
>> to
>> > > >> https://issues.apache.org/jira/browse/HDFS-6133, stating:
>> > > >>
>> > > >> "HDFS-6133 provides the ability to exclude a given directory from
>> the
>> > > >> HDFS load balancer, by setting the dfs.datanode.block-pinning.
>> enabled
>> > > >> property to true in your HDFS configuration and running the
>> following
>> > > >> hdfs command:
>> > > >>
>> > > >> $ sudo -u hdfs hdfs balancer -exclude /hbase"
>> > > >>
>> > > >> I checked the Balancer class in 2.7.2 and it does not have that
>> > > >> support, i.e. being able to exclude a path, it can only exclude
>> hosts.
>> > > >> That is also clear from HDFS-6133, which adds favoured nodes,
but
>> not
>> > > >> being able to exclude paths (which would be nice).
>> > > >>
>> > > >> HBASE-13021 mentions that this works in tandem with the HBase
>> favored
>> > > >> node feature, but that makes it much more complicated since you
have
>> > > >> to pin individual regions to nodes, instead of doing that wholesale.
>> > > >>
>> > > >> Where does the above in the HBase book come from, and what is
the
>> > > >> current state as far as you know?
>> > > >>
>> > > >> Cheers,
>> > > >> Lars
>> > > >>
>> > >
>> >
>>

Mime
View raw message