hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Segel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13323) Audit behavior heterogenous node capacity
Date Fri, 27 Mar 2015 16:42:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384114#comment-14384114
] 

Michael Segel  commented on HBASE-13323:
----------------------------------------

I don't know that you can say that the HDFS balancer is harmful. 
You're running HBase on top of HDFS so you have to live with the quirks of HDFS (balancer,
worm, etc ...) HDFS-7967 is a symptom of not thinking about the underlying FS and its impact
on HBase. 

Its important to think about the interaction of HDFS and HBase and to walk through the issue,
discussing the issue, long before your fingers touch the keyboard to pound out code. 

The reason I started to ask the questions is that there aren't really any good discussions
on this interaction. 



> Audit behavior heterogenous node capacity
> -----------------------------------------
>
>                 Key: HBASE-13323
>                 URL: https://issues.apache.org/jira/browse/HBASE-13323
>             Project: HBase
>          Issue Type: Task
>          Components: Balancer
>            Reporter: Nick Dimiduk
>              Labels: beginner
>
> From the thread "introducing nodes w/ more storage" (http://search-hadoop.com/m/DHED4azyle2),
we should have a look at what happens when nodes of varying data density are used in a single
cluster. The user would expect that nodes be filled according to their capacity, meaning an
"even distribution" looks like all nodes at the same pct use. This behavior is probably in
the intersection of hbase balancer and hdfs balancer. Probably this is made more complex by
recent HDFS features such as HDFS-5682.
> After investigation, let's fix it up to work better (if it's broken), and document the
behavior in our awesome book.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message