hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4365) Add a decent heuristic for region size
Date Sat, 10 Sep 2011 06:49:08 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101996#comment-13101996

Todd Lipcon commented on HBASE-4365:

bq. Load balancer currently doesn't balance the regions for any single table. We should introduce
a policy that does this.

That seems orthogonal, to me. If you have a single table in the cluster, then you need at
least as many regions as servers to make use of all of your servers.

If you have many tables, then yes, a per-table balancing might be useful (in some cases),
but it's the case regardless of whether we have a split size heuristic or manually set region

bq. It seems that the proposal favors not pre-splitting tables. If so, we need some solid
performance results to back the proposal.

Howso? I'm suggesting that we retain the MAX_REGION_SIZE parameter, if you want to manually
set it to some value, or set it to MAX_LONG and manually split. But, the default would be
this heuristic, which would work well for many use cases.

> Add a decent heuristic for region size
> --------------------------------------
>                 Key: HBASE-4365
>                 URL: https://issues.apache.org/jira/browse/HBASE-4365
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.94.0
>            Reporter: Todd Lipcon
> A few of us were brainstorming this morning about what the default region size should
be. There were a few general points made:
> - in some ways it's better to be too-large than too-small, since you can always split
a table further, but you can't merge regions currently
> - with HFile v2 and multithreaded compactions there are fewer reasons to avoid very-large
regions (10GB+)
> - for small tables you may want a small region size just so you can distribute load better
across a cluster
> - for big tables, multi-GB is probably best

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message