hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13323) Audit behavior heterogenous node capacity
Date Thu, 02 Apr 2015 22:57:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393664#comment-14393664

Andrew Purtell commented on HBASE-13323:

bq. You're running HBase on top of HDFS so you have to live with the quirks of HDFS (balancer,
worm, etc ...) 


bq. HDFS-7967 is a symptom of not thinking about the underlying FS and its impact on HBase.

Maybe you are thinking of another issue? I don't think HBase is involved on that one. 

If you can't run the balancer on a large cluster because it will mount a DoS on the NameNode,
it should be considered harmful. But that's an implementation problem that can be fixed.

I don't think the balancer is a problem for HBase as long as you're running a recent enough
HDFS including HDFS-6133. 

> Audit behavior heterogenous node capacity
> -----------------------------------------
>                 Key: HBASE-13323
>                 URL: https://issues.apache.org/jira/browse/HBASE-13323
>             Project: HBase
>          Issue Type: Task
>          Components: Balancer
>            Reporter: Nick Dimiduk
>              Labels: beginner
> From the thread "introducing nodes w/ more storage" (http://search-hadoop.com/m/DHED4azyle2),
we should have a look at what happens when nodes of varying data density are used in a single
cluster. The user would expect that nodes be filled according to their capacity, meaning an
"even distribution" looks like all nodes at the same pct use. This behavior is probably in
the intersection of hbase balancer and hdfs balancer. Probably this is made more complex by
recent HDFS features such as HDFS-5682.
> After investigation, let's fix it up to work better (if it's broken), and document the
behavior in our awesome book.

This message was sent by Atlassian JIRA

View raw message