hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8329) Limit compaction speed
Date Thu, 18 Apr 2013 17:17:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635359#comment-13635359
] 

Nick Dimiduk commented on HBASE-8329:
-------------------------------------

If I understand the tickets correctly, all three of them are addressing the same symptoms
via slightly different approaches. HBASE-5867 increases the threshold on the number of HFiles
that triggers a major compaction. HBASE-3743 introduces the idea of a major compaction manager
that orchestrates a rolling compaction across the cluster. This ticket introduces a local
compaction rate-limiter, configured on both time of day and IO throughput. [~aoxiang] does
that summary sound about right?

I personally like the idea of both the macro monitor (HBASE-3743) and the localized throttle
(this ticket). This patch would be improved by extracting out the throttling policy interface
with this peak+rate as one implementation. Based on my novice understanding of the storage
system, tweaking the threashold and HFiles count (HBASE-5867) is a bandaid specific to the
current default storage implementation. It will become less applicable as [~sershe]'s work
in modularization continues.

[~stack], [~nspiegelberg], [~larsgeorge], [~sershe] what's the right approach here?
                
> Limit compaction speed
> ----------------------
>
>                 Key: HBASE-8329
>                 URL: https://issues.apache.org/jira/browse/HBASE-8329
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction
>            Reporter: binlijin
>         Attachments: HBASE-8329-trunk.patch
>
>
> There is no speed or resource limit for compaction´╝îI think we should add this feature
especially when request burst.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message