hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8410) Basic quota support for namespaces
Date Fri, 21 Mar 2014 19:03:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943402#comment-13943402

Andrew Purtell commented on HBASE-8410:

bq. We are trying to stop users who might be running applications which might be accidentally
creating large number of tables or regions.

Understood. It would be good to reconsider if you really need something running in the regionserver
though. Installing a RegionObserver adds a certain (small) cost to every op even though you
are only sitting on a pre split hook. Rejecting splits in a pre split hook is messy as mentioned
above because you are not changing the higher level decisionmaking on whether to split or
not in the first place. Coordinating between multiple processes on the cluster requires either
meta scanning or the ZK mediated cache, which could be avoided if there is a single arbiter
running in the master. 

bq. ... accidentally creating large number of tables  ///

Can be handled in the master by intercepting table create requests in the namespace

bq. .. accidentally creating large number of [...] regions ...

On the Master manage RegionSplitPolicy details in the schema such that the user/application
cannot override the settings. For example, when through polling you discover a table is at
or just slightly over a high water mark,  disable region splits by setting “hbase.hregion.max.filesize”
in the table schema to the maximum value and set the split policy in the schema to ConstantSizeRegionSplitPolicy.

What this won't cover is interception of administrative split requests made directly to the
regionserver. However you could still handle this in the master by watching for when the number
of regions for a table is at or over a hard limit. Respond by offlining the table. A regionserver
won't process a split request for a region it is not serving. This achieves the objective
of stopping a runaway user or application.

> Basic quota support for namespaces
> ----------------------------------
>                 Key: HBASE-8410
>                 URL: https://issues.apache.org/jira/browse/HBASE-8410
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Francis Liu
>            Assignee: Vandana Ayyalasomayajula
>         Attachments: HBASE-8410.docx, HBASE-8410_trunk_10.patch, HBASE-8410_trunk_10.patch,
HBASE-8410_trunk_11.patch, HBASE-8410_trunk_12.patch, HBASE-8410_trunk_13.patch, HBASE-8410_trunk_14.patch,
HBASE-8410_trunk_15.patch, HBASE-8410_trunk_2.patch, HBASE-8410_trunk_3.patch, HBASE-8410_trunk_4.patch,
HBASE-8410_trunk_4.patch, HBASE-8410_trunk_5.patch, HBASE-8410_trunk_6.patch, HBASE-8410_trunk_7.patch,
HBASE-8410_trunk_8.patch, HBASE-8410_trunk_9.patch, HBASE_8410.patch, HBASE_8410_1_trunk.patch
> This task involves creating an observer which provides basic quota support to namespaces
in terms of (1) number of tables and (2) number of regions. The quota support can be enabled
by setting:
> <property>
>     <name>hbase.coprocessor.region.classes</name>
>     <value>org.apache.hadoop.hbase.namespace.NamespaceController</value>
> </property>
> <property>
>     <name>hbase.coprocessor.master.classes</name>
>     <value>org.apache.hadoop.hbase.namespace.NamespaceController</value>
> </property>
> in the hbase-site.xml.
> To add quotas to namespace, while creating namespace properties need to be added.
> Examples:
> 1. namespace_create 'ns1', {'hbase.namespace.quota.maxregion'=>'10'}
> 2. 1. namespace_create 'ns2', {'hbase.namespace.quota.maxtables'=>'2'}, {'hbase.namespace.quota.maxregion'=>'5'}
> The quotas can be modified/added to namespace at any point of time. 

This message was sent by Atlassian JIRA

View raw message