hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17515) Reduce memory footprint of RegionLoads kept by StochasticLoadBalancer
Date Thu, 26 Jan 2017 18:36:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840189#comment-15840189
] 

stack commented on HBASE-17515:
-------------------------------

bq. Will look at Instrumentation result to see the size before reduction.

That is not the point and this is after the fact anyways. The work is done. Outstanding items
(that should have been done as preamble) are, if looking to optimize (as is the predicate
to this JIRA), is (whatever the size) too much to keep around for single input to the balancer?
Is the amount of mem kept even an issue in the scheme of things? Are there higher priority
fat to trim? If it is too much (and it seems so), can we take a different approach, say, by
boiling down the data size by using a purposed library.

Finally, our contributors need to get quality engineering input/direction and examples to
follow so that when they get to do it for themselves, the bar is set higher. If unable to
provide this, work elsewhere were you are able to do this.

> Reduce memory footprint of RegionLoads kept by StochasticLoadBalancer
> ---------------------------------------------------------------------
>
>                 Key: HBASE-17515
>                 URL: https://issues.apache.org/jira/browse/HBASE-17515
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Tim Brown
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: 17515.branch-1.v2.txt, 17515.v1.txt, 17515.v2.txt
>
>
> Currently StochasticLoadBalancer uses the following fields of RegionLoad :
> {code}
>       return rl.getReadRequestsCount();
>       return rl.getWriteRequestsCount();
>       return rl.getMemStoreSizeMB();
>       return rl.getStorefileSizeMB();
> {code}
> However, RegionLoad refers to ClusterStatusProtos.RegionLoad which has 18 fields.
> This means we keep hbase.master.balancer.stochastic.numRegionLoadsToRemember (default
value 15) RegionLoad's in memory but only use a small portion of them.
> This JIRA is to consider creating POJO which wraps the above 4 fields so that the memory
footprint can be lowered.
> Lowering memory footprint would allow wider sliding window to be configured for load
balancer while limiting the impact to heap consumption.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message