hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kahlil Oppenheimer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17707) New More Accurate Table Skew cost function/generator
Date Sun, 19 Mar 2017 17:20:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931834#comment-15931834

Kahlil Oppenheimer commented on HBASE-17707:

[~enis] [~tedyu] the new table skew cost function is actually guaranteed to be within the
[0-1] range (and this behavior is even unit tested!). The cost function does not dominate
over other cost functions because it is out of the [0-1] range. Instead, I debugged the breaking
test and found that the issue is that the region replica host cost function can produce very
small values when there are a lot of regions. In my testing, I found that for some medium-large
cluster sizes, the cost function can produce values as small as 2.6 x 10^(-6). Sadly, this
means that even with a weight of 5000 (which is what is set in the test), the "soft" requirement
of having no two region replicas hosted on the same machine when it is avoidable is not met
because the cost function has too small a contribution (even with this high weight). Instead,
my latest patch updates the region replica cost function to give it a minimum value (.1) for
any amount of co-hosted replicas. This makes it so that if two regions replicas are placed
on the same host, the cost will be at least .1 (whether or not there are 5 or 1,000,000 regions
in the cluster). This better enforces the "soft" constraint as it makes sure that no other
cost functions can overpower the region replica host cost function.

> New More Accurate Table Skew cost function/generator
> ----------------------------------------------------
>                 Key: HBASE-17707
>                 URL: https://issues.apache.org/jira/browse/HBASE-17707
>             Project: HBase
>          Issue Type: New Feature
>          Components: Balancer
>    Affects Versions: 1.2.0
>         Environment: CentOS Derivative with a derivative of the 3.18.43 kernel. HBase
on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.
>            Reporter: Kahlil Oppenheimer
>            Assignee: Kahlil Oppenheimer
>            Priority: Minor
>             Fix For: 2.0
>         Attachments: HBASE-17707-00.patch, HBASE-17707-01.patch, HBASE-17707-02.patch,
HBASE-17707-03.patch, HBASE-17707-04.patch, HBASE-17707-05.patch, HBASE-17707-06.patch, HBASE-17707-07.patch,
HBASE-17707-08.patch, HBASE-17707-09.patch, HBASE-17707-11.patch, HBASE-17707-11.patch, test-balancer2-13617.out
> This patch includes new version of the TableSkewCostFunction and a new TableSkewCandidateGenerator.
> The new TableSkewCostFunction computes table skew by counting the minimal number of region
moves required for a given table to perfectly balance the table across the cluster (i.e. as
if the regions from that table had been round-robin-ed across the cluster). This number of
moves is computer for each table, then normalized to a score between 0-1 by dividing by the
number of moves required in the absolute worst case (i.e. the entire table is stored on one
server), and stored in an array. The cost function then takes a weighted average of the average
and maximum value across all tables. The weights in this average are configurable to allow
for certain users to more strongly penalize situations where one table is skewed versus where
every table is a little bit skewed. To better spread this value more evenly across the range
0-1, we take the square root of the weighted average to get the final value.
> The new TableSkewCandidateGenerator generates region moves/swaps to optimize the above
TableSkewCostFunction. It first simply tries to move regions until each server has the right
number of regions, then it swaps regions around such that each region swap improves table
skew across the cluster.
> We tested the cost function and generator in our production clusters with 100s of TBs
of data and 100s of tables across dozens of servers and found both to be very performant and

This message was sent by Atlassian JIRA

View raw message