hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9528) Adaptive compaction
Date Fri, 13 Sep 2013 10:32:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13766387#comment-13766387
] 

stack commented on HBASE-9528:
------------------------------

Sounds like good idea to me.

How you think it would work?  When a region wants to compact, it would send its desire along
w/ stats on the compaction it wants to run (bytes, files)  to a central location.  Then the
coordinator, or 'central planner' would give out who could compact when?  When a compaction
starts/stops, it would let the coordinator know.

The Master should be the coordinator.  Would the desire to compact come in on the back of
the heartbeats?  (We should try and get 'load' off the heartbeats and have 'load' instead
come into the server via metrics).  Master would need to keep these stats somewhere?  In a
system table?

Should we call it "central planning compaction" rather than adaptive?

Good on you [~xieliang007]
                
> Adaptive compaction
> -------------------
>
>                 Key: HBASE-9528
>                 URL: https://issues.apache.org/jira/browse/HBASE-9528
>             Project: HBase
>          Issue Type: Brainstorming
>    Affects Versions: 0.98.0
>            Reporter: Liang Xie
>
> Currently, the compaction policy granularity is based on single machine. we had a thought
that introduce a new cluster granularity decision, such that we could improve those case per
cluster running status:
> 1) many nodes are compacting aggressive, we call it cluster compaction storm, we should
throttle it.
> 2) do more compaction if low traffic in current cluster(similar with off-peak feature),
not limit by config timerange(like off-peak timerange), just trigger by load or qps or other
stuff.
> comments? thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message