hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-18151) Improvement on Histogram
Date Fri, 02 Jun 2017 11:16:04 GMT
Phil Yang created HBASE-18151:

             Summary: Improvement on Histogram
                 Key: HBASE-18151
                 URL: https://issues.apache.org/jira/browse/HBASE-18151
             Project: HBase
          Issue Type: Improvement
            Reporter: Phil Yang

There are some bugs and some places we can improvement in the new native Histogram.

The default maxExpected value in HistogramImpl is Integer.MAX_VALUE << 2 (https://github.com/apache/hbase/blob/master/hbase-metrics/src/main/java/org/apache/hadoop/hbase/metrics/impl/HistogramImpl.java#L41
) which is overflow... and will be regarded as -4... And binsMax pf FastLongHistogram will
be 255(by the reason of this bug) which is too low (In FastLongHistogram#L75 it says the expected
binsMax is 1000? I think it may be also too low. Even if there is no overflow, Integer.MAX_VALUE*4
is too large?

We may need pass the right expected min/max value when we use it because the value may be
different in scenarios. And the default value should not be 1000 or Integer.MAX_VALUE*4.

There is an assumption that the value is in uniform distribution. But if the value is latency
which is the most use case, the distribution is not uniform. The lower the value, the higher
the probability.

Because of these reasons, the 75/90/99 or other percentages may have big error than we expected.
We need find a way to reduce the error.

This message was sent by Atlassian JIRA

View raw message