hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8410) Basic quota support for namespaces
Date Thu, 20 Mar 2014 23:41:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13942529#comment-13942529

Andrew Purtell commented on HBASE-8410:

It is relatively messy if we treat a region quota as an absolute and actively manage it. A
coprocessor can sit in the preSplit hook and reject split transactions. However, this will
not prevent the regionserver from deciding to try and split the region again (and again).
I'm not sure anything will stop the resulting log spam but raising the quota or an online
schema update. (See below.)

Another issue is all of the meta scanning that might be involved if the namespace auditor
is running on all regionservers as well as the master. There's been some discussion about
mitigating this on RB, but the issue could be side stepped.

How about accepting that the region limit is a soft limit. Then instead of running on every
regionserver the auditor needs only run on the master, periodically poll meta for if a namespace
has crossed the threshold, and then issue online schema updates to update the split policy
and/or split parameters for the table or all tables in the namespace. Would this work? Would
avoid the two issues mentioned above I think.

> Basic quota support for namespaces
> ----------------------------------
>                 Key: HBASE-8410
>                 URL: https://issues.apache.org/jira/browse/HBASE-8410
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Francis Liu
>            Assignee: Vandana Ayyalasomayajula
>         Attachments: HBASE-8410.docx, HBASE-8410_trunk_10.patch, HBASE-8410_trunk_10.patch,
HBASE-8410_trunk_11.patch, HBASE-8410_trunk_12.patch, HBASE-8410_trunk_13.patch, HBASE-8410_trunk_14.patch,
HBASE-8410_trunk_15.patch, HBASE-8410_trunk_2.patch, HBASE-8410_trunk_3.patch, HBASE-8410_trunk_4.patch,
HBASE-8410_trunk_4.patch, HBASE-8410_trunk_5.patch, HBASE-8410_trunk_6.patch, HBASE-8410_trunk_7.patch,
HBASE-8410_trunk_8.patch, HBASE-8410_trunk_9.patch, HBASE_8410.patch, HBASE_8410_1_trunk.patch
> This task involves creating an observer which provides basic quota support to namespaces
in terms of (1) number of tables and (2) number of regions. The quota support can be enabled
by setting:
> <property>
>     <name>hbase.coprocessor.region.classes</name>
>     <value>org.apache.hadoop.hbase.namespace.NamespaceController</value>
> </property>
> <property>
>     <name>hbase.coprocessor.master.classes</name>
>     <value>org.apache.hadoop.hbase.namespace.NamespaceController</value>
> </property>
> in the hbase-site.xml.
> To add quotas to namespace, while creating namespace properties need to be added.
> Examples:
> 1. namespace_create 'ns1', {'hbase.namespace.quota.maxregion'=>'10'}
> 2. 1. namespace_create 'ns2', {'hbase.namespace.quota.maxtables'=>'2'}, {'hbase.namespace.quota.maxregion'=>'5'}
> The quotas can be modified/added to namespace at any point of time. 

This message was sent by Atlassian JIRA

View raw message