hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francis Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
Date Tue, 09 Sep 2014 03:41:32 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126542#comment-14126542

Francis Liu commented on HBASE-11165:

I'm concerned about this notion of a split meta colocated with a split master function also.
Split meta on its own is one thing. Also splitting the master as described smacks of the dynamic
subtree partitioning of Ceph's MDS, which has never been stable at scale as far as I know,
and there's no stability in sight there either.
>From what I understand "splitting meta" supercedes "collocating the meta" as having both
together would be counterintuitive. With splitting meta I think it is understood enough that
code plus documentation should be enough?

There's also the issue that Francis and crew want to run a version of HBase in production
today. I don't see the multi-master alternative as viable before 2.0, or perhaps 1.1 I suppose,
but not 1.0.x. Split meta is conceivably something that could be backported like ZK-less assignment*
was, although no doubt risky and would need to be carefully done, hidden behind a default-off
For our use case we don't have data yet to motivate sharded master or HA master. For now it
seems splitting meta will address our short-term scaling use case.

> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>                 Key: HBASE-11165
>                 URL: https://issues.apache.org/jira/browse/HBASE-11165
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: stack
>         Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569" and comments
on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe
even 50M later.  This issue is about discussing how we will do that (or if not 50M on a cluster,
how otherwise we can attain same end).
> More detail to follow.

This message was sent by Atlassian JIRA

View raw message