hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8329) Limit compaction speed
Date Thu, 18 Apr 2013 17:17:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635359#comment-13635359

Nick Dimiduk commented on HBASE-8329:

If I understand the tickets correctly, all three of them are addressing the same symptoms
via slightly different approaches. HBASE-5867 increases the threshold on the number of HFiles
that triggers a major compaction. HBASE-3743 introduces the idea of a major compaction manager
that orchestrates a rolling compaction across the cluster. This ticket introduces a local
compaction rate-limiter, configured on both time of day and IO throughput. [~aoxiang] does
that summary sound about right?

I personally like the idea of both the macro monitor (HBASE-3743) and the localized throttle
(this ticket). This patch would be improved by extracting out the throttling policy interface
with this peak+rate as one implementation. Based on my novice understanding of the storage
system, tweaking the threashold and HFiles count (HBASE-5867) is a bandaid specific to the
current default storage implementation. It will become less applicable as [~sershe]'s work
in modularization continues.

[~stack], [~nspiegelberg], [~larsgeorge], [~sershe] what's the right approach here?
> Limit compaction speed
> ----------------------
>                 Key: HBASE-8329
>                 URL: https://issues.apache.org/jira/browse/HBASE-8329
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction
>            Reporter: binlijin
>         Attachments: HBASE-8329-trunk.patch
> There is no speed or resource limit for compaction´╝îI think we should add this feature
especially when request burst.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message