Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 28 Apr 2017 12:26:04 +0000 (UTC)
From: "Josh Elser (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13016204.1477671958000.78249.1493382364344@Atlassian.JIRA>
In-Reply-To: <JIRA.13016204.1477671958000@Atlassian.JIRA>
References: <JIRA.13016204.1477671958000@Atlassian.JIRA> <JIRA.13016204.1477671958571@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HBASE-16961) FileSystem Quotas
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 28 Apr 2017 12:26:09 -0000


    [ https://issues.apache.org/jira/browse/HBASE-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988733#comment-15988733 ] 

Josh Elser commented on HBASE-16961:
------------------------------------

Sorry, for the delayed reply and thanks for taking a look [~apurtell].

bq. 1. About the optional MasterObserver for automatically deleting quotas when the table is deleted. Why do we not want this in general? Make it a part of core master, optionally disabled?

Mostly, this came in later; I think that's why I optionally enabled it. I don't have a good reason that it should be disable by default.

bq. 2. Under what circumstances is NO_WRITES_COMPACTIONS advisable instead of NO_WRITES? 

It comes down to how strict administrators want to be with disk usage. As we move towards a reality where space use is closely tracked, we might also want to avoid temporary spikes in space usage for the compaction's file set (e.g. a compaction of three files, we start out with space use of those three files, but then create a new file which would be similar in size. So, we'd actually have ~2x the space taken, until the original 3 files are removed).

bq. 3. How would quotas and system recovery actions interact? Are quota checks bypassed for actions taken by a superuser?

I actually haven't spent much time investigating this. I'm not sure if I've added short-circuits for hbase.superusers. Did you have some specific commands in mind WRT "system recovery actions"? I could probably whip up a unit test without much hassle.

bq. Yes, this could be a problem, if we are trying to apply quotas to more than one table as part of activity to bring something runaway under control. Not urgent, but let's make sure there's a follow up JIRA for this. 

Will create and link here.

> FileSystem Quotas
> -----------------
>
>                 Key: HBASE-16961
>                 URL: https://issues.apache.org/jira/browse/HBASE-16961
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>         Attachments: hbase-quota-test.sh
>
>
> Umbrella issue for tracking the filesystem utilization of HBase data, defining quotas on that utilization, and enforcement when utilization exceeds the limits of the quota.
> At a high level: we can define quotas on tables and namespaces. Region size is computed by RegionServers and sent to the Master. The Master inspects the sizes of Regions, rolling up to table and namespace sizes. Defined quotas in the quota table are evaluated given the computed sizes, and, for those tables/namespaces violating the quota, RegionServers are informed to take some action to limit any further filesystem growth by that table/namespace.
> Discuss: https://lists.apache.org/thread.html/66a4b0c3725b5cbdd61dd6111c43847adaeef7b7da5f4cd045df30ef@%3Cdev.hbase.apache.org%3E
> Design Doc: http://home.apache.org/~elserj/hbase/FileSystemQuotasforApacheHBase.pdf or https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)