hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-17462) Investigate using sliding window for read/write request costs in StochasticLoadBalancer
Date Fri, 13 Jan 2017 18:27:26 GMT
Ted Yu created HBASE-17462:
------------------------------

             Summary: Investigate using sliding window for read/write request costs in StochasticLoadBalancer
                 Key: HBASE-17462
                 URL: https://issues.apache.org/jira/browse/HBASE-17462
             Project: HBase
          Issue Type: Improvement
            Reporter: Ted Yu


In the thread, http://search-hadoop.com/m/HBase/YGbbyUZKXWALkX1, Timothy was asking whether
the read/write request costs in StochasticLoadBalancer should be calculated as rates.

This makes sense since read / write load on region server tends to fluctuate over time. Using
sliding window would reflect more recent trend in read / write load.

Some factors to consider:

The data structure used by StochasticLoadBalancer should be concise. The
number of regions in a cluster can be expected to approach 1 million. We
cannot afford to store long history of read / write requests in master.

Efficiency of cost calculation should be high - there're many cost
functions the balancer goes through, it is expected for each cost function
to return quickly. Otherwise we would not come up with proper region
movement plan(s) in time.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message