hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Helmling (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HBASE-16958) Balancer recomputes block distributions every time balanceCluster() runs
Date Thu, 27 Oct 2016 23:58:58 GMT

     [ https://issues.apache.org/jira/browse/HBASE-16958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Gary Helmling resolved HBASE-16958.
       Resolution: Duplicate
         Assignee:     (was: Gary Helmling)
    Fix Version/s:     (was: 1.3.0)

I re-opened HBASE-16570 to fix the issue that is described here.

> Balancer recomputes block distributions every time balanceCluster() runs
> ------------------------------------------------------------------------
>                 Key: HBASE-16958
>                 URL: https://issues.apache.org/jira/browse/HBASE-16958
>             Project: HBase
>          Issue Type: Bug
>          Components: Balancer
>            Reporter: Gary Helmling
> The change in HBASE-16570 modified the balancer to compute block distributions in parallel
with a pool of 5 threads.  However, because it does this every time Cluster is instantiated,
it effectively bypasses the cache of block locations added in HBASE-14473:
> In the LoadBalancer.balanceCluster() implementations (in StochasticLoadBalancer, SimpleLoadBalancer),
we create a new Cluster instance.
> In Cluster.<init>, we call registerRegion() on every HRegionInfo.
> In registerRegion(), we do the following:
> {code}
>         regionLocationFutures.set(regionIndex,
>             regionFinder.asyncGetBlockDistribution(region));
> {code}
> Then, back in Cluster.<init> we do a get() on each ListenableFuture in a loop.
> So while we are doing the calls to get block locations in parallel with 5 threads, we're
recomputing them every time balanceCluster() is called and not taking advantage of the cache
at all.

This message was sent by Atlassian JIRA

View raw message