hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)
Date Tue, 20 May 2014 02:26:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002716#comment-14002716

Andrew Purtell commented on HBASE-11165:

bq. What do you think, how expensive is to resize memstore and how often we may need it? Another
thing - how it might affect 99th latency. 

If we are using a lower flush threshold for some regions, then we'd flush more than otherwise
if a burst of writes comes in, and increase compaction activity. Would be important to notice
and adjust for workload changes. If we are lazily allocating mslabs also then we could trigger
GC activity as those are large objects. It might be worth keeping a pool of mslabs around
for use by memstores as needed, in aggregate still a lesser allocation than today, the size
of the pool adjusted by heuristics. 

> Scaling so cluster can host 1M regions and beyond (50M regions?)
> ----------------------------------------------------------------
>                 Key: HBASE-11165
>                 URL: https://issues.apache.org/jira/browse/HBASE-11165
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: stack
> This discussion issue comes out of "Co-locate Meta And Master HBASE-10569" and comments
on the doc posted there.
> A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M regions maybe
even 50M later.  This issue is about discussing how we will do that (or if not 50M on a cluster,
how otherwise we can attain same end).
> More detail to follow.

This message was sent by Atlassian JIRA

View raw message