hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francis Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
Date Fri, 15 Aug 2014 22:31:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14099283#comment-14099283

Francis Liu commented on HBASE-11165:

I haven't read the full discussion, so sorry if I missed this piece.
does splitting meta means having multiple master each one handing its own meta and its own
set of RS?
It does not have to be. So far we can see there are gains to be made in scalability by spliting
meta so that it is served by multiple RS. We will do the work to motivate multi-master and
justify the complexity if need be.

otherwise we go back as before, the idea of having meta colocated with the master was have
operation like assignment, or disable/enable, create/delete interact with "local data" and
avoid complexity in handling failure when interacting with other machines.
I don't see why collocation is required to have the features you mentioned. IMHO all you need
is the mutations/locking/etc to be centrally managed (ie all updates to meta is sent to the
master), the meta itself can be anywhere. So I'd argue that central management is needed,
collocation is not required. Please correct me if I'm missing something here.

> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>                 Key: HBASE-11165
>                 URL: https://issues.apache.org/jira/browse/HBASE-11165
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: stack
>         Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison.pdf,
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569" and comments
on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe
even 50M later.  This issue is about discussing how we will do that (or if not 50M on a cluster,
how otherwise we can attain same end).
> More detail to follow.

This message was sent by Atlassian JIRA

View raw message