hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-16961) FileSystem Quotas
Date Fri, 24 Mar 2017 21:26:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941163#comment-15941163
] 

Josh Elser edited comment on HBASE-16961 at 3/24/17 9:26 PM:
-------------------------------------------------------------

[~apurtell] and others, here's a [little script|https://issues.apache.org/jira/secure/attachment/12860455/hbase-quota-test.sh]
you can walk through for a general purpose scenario of setting a space quota on a namespace.

Setup-wise:

# Build the HEAD of HBASE-16961
# Set {{hbase.quota.enabled=true}} in hbase-site.xml

Optionally: you can also tweak some of the chore properties to see more responsiveness:

{noformat}
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.period</name>
    <value>60000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.delay</name>
    <value>1000</value>
  </property>
{noformat}


was (Author: elserj):
[~apurtell] and others, here's a little script you can walk through for a general purpose
scenario of setting a space quota on a namespace.

Setup-wise:

# Build the HEAD of HBASE-16961
# Set {{hbase.quota.enabled=true}} in hbase-site.xml

Optionally: you can also tweak some of the chore properties to see more responsiveness:

{noformat}
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.period</name>
    <value>60000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.delay</name>
    <value>1000</value>
  </property>
{noformat}

> FileSystem Quotas
> -----------------
>
>                 Key: HBASE-16961
>                 URL: https://issues.apache.org/jira/browse/HBASE-16961
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>         Attachments: hbase-quota-test.sh
>
>
> Umbrella issue for tracking the filesystem utilization of HBase data, defining quotas
on that utilization, and enforcement when utilization exceeds the limits of the quota.
> At a high level: we can define quotas on tables and namespaces. Region size is computed
by RegionServers and sent to the Master. The Master inspects the sizes of Regions, rolling
up to table and namespace sizes. Defined quotas in the quota table are evaluated given the
computed sizes, and, for those tables/namespaces violating the quota, RegionServers are informed
to take some action to limit any further filesystem growth by that table/namespace.
> Discuss: https://lists.apache.org/thread.html/66a4b0c3725b5cbdd61dd6111c43847adaeef7b7da5f4cd045df30ef@%3Cdev.hbase.apache.org%3E
> Design Doc: http://home.apache.org/~elserj/hbase/FileSystemQuotasforApacheHBase.pdf or
https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message