hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaxiang sun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15324) Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger unexpected split
Date Wed, 09 Nov 2016 19:25:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15651817#comment-15651817
] 

huaxiang sun commented on HBASE-15324:
--------------------------------------

Hi [~liyu], have one question about the fix. Why the overflow is only checked when (this.jitterRate
> EPSILON). I modified a bit of the test case, it can get a minus desiredMaxFileSize (overflow).
Should the overflow be checked for all the cases? Thanks.
{code}
  @Test
  public void testConstantSizePolicyWithJitter() throws IOException {
    conf.set(HConstants.HBASE_REGION_SPLIT_POLICY_KEY,
      ConstantSizeRegionSplitPolicy.class.getName());
    htd.setMaxFileSize(Long.MAX_VALUE);
    boolean positiveJitter = false;
    ConstantSizeRegionSplitPolicy policy = null;
    long value = 0;
    while (value >= 0) {
      policy = (ConstantSizeRegionSplitPolicy) RegionSplitPolicy.create(mockRegion, conf);
      //positiveJitter = policy.positiveJitterRate();
      value = policy.getDesiredMaxFileSize();
      if (value < 0) {
        System.out.println(policy.getDesiredMaxFileSize());
        System.out.println(policy.getJitterRate());
      }
    }

    // add a store
    HStore mockStore = Mockito.mock(HStore.class);
    Mockito.doReturn(2000L).when(mockStore).getSize();
    Mockito.doReturn(true).when(mockStore).canSplit();
    stores.add(mockStore);
    // Jitter shouldn't cause overflow when HTableDescriptor.MAX_FILESIZE set to Long.MAX_VALUE
    assertFalse(policy.shouldSplit());
  }
{code}

For the failed case, values are:
-9223365302346055681 (getDesiredMaxFileSize)
7.301568984985352E-7 (jitter rate)

> Jitter may cause desiredMaxFileSize overflow in ConstantSizeRegionSplitPolicy and trigger
unexpected split
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15324
>                 URL: https://issues.apache.org/jira/browse/HBASE-15324
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.1.3
>            Reporter: Yu Li
>            Assignee: Yu Li
>             Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8
>
>         Attachments: HBASE-15324.patch, HBASE-15324_v2.patch, HBASE-15324_v3.patch, HBASE-15324_v3.patch
>
>
> We introduce jitter for region split decision in HBASE-13412, but the following line
in {{ConstantSizeRegionSplitPolicy}} may cause long value overflow if MAX_FILESIZE is specified
to Long.MAX_VALUE:
> {code}
> this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() - 0.5D) *
jitter);
> {code}
> In our case we specify MAX_FILESIZE to Long.MAX_VALUE to prevent target region to split.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message