hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Qiang Tian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12451) IncreasingToUpperBoundRegionSplitPolicy may cause unnecessary region splits in rolling update of cluster
Date Fri, 14 Nov 2014 09:46:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14212065#comment-14212065
] 

Qiang Tian commented on HBASE-12451:
------------------------------------

bq. Add a minimum split size will break the first goal, right? 
it is configurable, we can set the default minimum size to 0, the calculation will ignore
the minimum size in this case:(Pseudocode)
{code}
    long splitSize = tableRegionsCount == 0 || tableRegionsCount > 100 ? getDesiredMaxFileSize():
      Math.min(getDesiredMaxFileSize(),
        this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount);

    long minmumSize = conf.getLong("hbase.hregion.split.minimum.size", 0);
    return minmumSize > 0 ? Math.max(splitSize, minmumSize) : splitSize;
  }
{code}

bq. And actually, there is a "hbase.increasing.policy.initial.size" configuration in IncreasingToUpperBoundRegionSplitPolicy
which specify the initial(minimum) split size. 

beside initial.size, tableRegionsCount is also a variable.

bq. And if you do not like the curve of split size, I think we should introduce a new split
policy instead?
I do not mean I do not like it, it looks to me the case described in the jira is not a common
case? so a simple fix can resolve it quickly...if there are other issues I am open for ideas
:-)




> IncreasingToUpperBoundRegionSplitPolicy may cause unnecessary region splits in rolling
update of cluster
> --------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-12451
>                 URL: https://issues.apache.org/jira/browse/HBASE-12451
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBASE-12451-v1.diff
>
>
> Currently IncreasingToUpperBoundRegionSplitPolicy is the default region split policy.
In this policy, split size is the number of regions that are on this server that all are of
the same table, cubed, times 2x the region flush size.
> But when unloading regions of a regionserver in a cluster using region_mover.rb, the
number of regions that are on this server that all are of the same table will decrease, and
the split size will decrease too, which may cause the left region split in the regionsever.
Region Splits also happens when loading regions of a regionserver in a cluster. 
> A improvment may set a minimum split size in IncreasingToUpperBoundRegionSplitPolicy
> Suggestions are welcomed. Thanks~



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message