hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhangduo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-8329) Limit compaction speed
Date Wed, 14 Jan 2015 00:38:36 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

zhangduo updated HBASE-8329:
    Release Note: 
Adds compaction throughput limit mechanism(the word "throttle" is already used when choosing
compaction thread pool, so use a different word here to avoid ambiguity). Default is org.apache.hadoop.hbase.regionserver.compactions.DefaultThroughputController,
will limit throughput as follow:
1. In off peak hours, use a fixed limitation "hbase.hstore.compaction.throughput.offpeak"
(default 40MB/sec)
2. In normal hours, the limitation is tuned between "hbase.hstore.compaction.throughput.lower.bound"(default
20MB/sec) and "hbase.hstore.compaction.throughput.higher.bound"(default 10MB/sec), using the
formula "lower + (higer - lower) * param" where param is in range [0.0, 1.0] and calculate
based on store files count on this regionserver.
3. If some stores have too many store files(storefilesCount > blockingFileCount), then
there is no limitation no matter peak or off peak.
You can set "hbase.regionserver.throughput.controller" to org.apache.hadoop.hbase.regionserver.compactions.NoLimitThroughputController
to disable throughput controlling.
And we have implemented ConfigurationObserver which means you can change all configurations
above and do not need to restart cluster.

> Limit compaction speed
> ----------------------
>                 Key: HBASE-8329
>                 URL: https://issues.apache.org/jira/browse/HBASE-8329
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction
>            Reporter: binlijin
>            Assignee: zhangduo
>             Fix For: 2.0.0, 1.1.0
>         Attachments: HBASE-8329-10.patch, HBASE-8329-11.patch, HBASE-8329-12.patch, HBASE-8329-2-trunk.patch,
HBASE-8329-3-trunk.patch, HBASE-8329-4-trunk.patch, HBASE-8329-5-trunk.patch, HBASE-8329-6-trunk.patch,
HBASE-8329-7-trunk.patch, HBASE-8329-8-trunk.patch, HBASE-8329-9-trunk.patch, HBASE-8329-trunk.patch,
HBASE-8329_13.patch, HBASE-8329_14.patch
> There is no speed or resource limit for compaction´╝îI think we should add this feature
especially when request burst.

This message was sent by Atlassian JIRA

View raw message