hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4365) Add a decent heuristic for region size
Date Thu, 23 Feb 2012 23:09:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215176#comment-13215176
] 

stack commented on HBASE-4365:
------------------------------

@Lars You want to put an upper bound on the number of regions?

I think if we do power of three, we'll lose some of the benefit J-D sees above; we'll fan
out the regions slower.

Do you want to put an upper bound on the number of regions per regionserver for a table? 
Say, three?  As in, when we get to three regions on a server, just scoot the split size up
to the maximum.  So, given a power of two, we'd split on first flush, then the next split
would happen at (2*2*128M) 512M, then 9*128M=1.2G and thereafter we'd split at the max, say
10G?

Or should we just commit this for now and do more in another patch?
                
> Add a decent heuristic for region size
> --------------------------------------
>
>                 Key: HBASE-4365
>                 URL: https://issues.apache.org/jira/browse/HBASE-4365
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.92.1, 0.94.0
>            Reporter: Todd Lipcon
>            Priority: Critical
>              Labels: usability
>         Attachments: 4365-v2.txt, 4365.txt
>
>
> A few of us were brainstorming this morning about what the default region size should
be. There were a few general points made:
> - in some ways it's better to be too-large than too-small, since you can always split
a table further, but you can't merge regions currently
> - with HFile v2 and multithreaded compactions there are fewer reasons to avoid very-large
regions (10GB+)
> - for small tables you may want a small region size just so you can distribute load better
across a cluster
> - for big tables, multi-GB is probably best

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message