hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
Date Wed, 03 Sep 2014 16:58:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120086#comment-14120086
] 

stack commented on HBASE-11165:
-------------------------------

bq.  I mean, seems tricky to make it be the same code path?

Yeah. One of the code paths will 'suffer' neglect.

[~mantonov] I like your "cold backup -> warm -> hot -> active-active".... let me
try and put together a bit of a summary so far on findings and arguments.


bq. ...and reduce usage of memory

Yeah, we'll have to go this route if we are trying to keep state of a big cluster in heap.
 Could work on making the representation more compact.  You arguing for single meta region
[~octo47] then? There is also the on-hdfs size to consider (write-amplification) and the r/w
i/os.

> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>
>                 Key: HBASE-11165
>                 URL: https://issues.apache.org/jira/browse/HBASE-11165
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: stack
>         Attachments: HBASE-11165.zip, Region Scalability test.pdf, zk_less_assignment_comparison_2.pdf
>
>
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569" and comments
on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe
even 50M later.  This issue is about discussing how we will do that (or if not 50M on a cluster,
how otherwise we can attain same end).
> More detail to follow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message