hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ndimi...@apache.org
Subject [2/2] hbase git commit: updating docs from master
Date Sun, 11 Jun 2017 01:03:46 GMT
updating docs from master

Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/0dab4ad2
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/0dab4ad2
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/0dab4ad2

Branch: refs/heads/branch-1.1
Commit: 0dab4ad2a03fb64530582329b773fa337301a7a1
Parents: 7a823b4
Author: Nick Dimiduk <ndimiduk@apache.org>
Authored: Sat Jun 10 17:49:35 2017 -0700
Committer: Nick Dimiduk <ndimiduk@apache.org>
Committed: Sat Jun 10 17:49:35 2017 -0700

 src/main/asciidoc/_chapters/external_apis.adoc |  9 ++-
 src/main/asciidoc/_chapters/faq.adoc           |  3 +
 src/main/asciidoc/_chapters/hbase_mob.adoc     |  4 +-
 src/main/asciidoc/_chapters/ops_mgt.adoc       | 80 ++++++++++++++++++++-
 src/main/asciidoc/_chapters/orca.adoc          |  6 +-
 src/main/asciidoc/_chapters/upgrading.adoc     |  2 +-
 6 files changed, 96 insertions(+), 8 deletions(-)

diff --git a/src/main/asciidoc/_chapters/external_apis.adoc b/src/main/asciidoc/_chapters/external_apis.adoc
index 556c4e0..2f85461 100644
--- a/src/main/asciidoc/_chapters/external_apis.adoc
+++ b/src/main/asciidoc/_chapters/external_apis.adoc
@@ -225,14 +225,17 @@ creation or mutation, and `DELETE` for deletion.
-|Get the value of a single row. Values are Base-64 encoded.
+|Get all columns of a single row. Values are Base-64 encoded. This requires the "Accept"
request header with a type that can hold multiple columns (like xml, json or protobuf).
 |curl -vi -X GET \
   -H "Accept: text/xml" \
-curl -vi -X GET \
+|Get the value of a single column. Values are Base-64 encoded.
+|curl -vi -X GET \
   -H "Accept: text/xml" \

diff --git a/src/main/asciidoc/_chapters/faq.adoc b/src/main/asciidoc/_chapters/faq.adoc
index 7bffe0e..9034d4b 100644
--- a/src/main/asciidoc/_chapters/faq.adoc
+++ b/src/main/asciidoc/_chapters/faq.adoc
@@ -44,6 +44,9 @@ How can I find examples of NoSQL/HBase?::
 What is the history of HBase?::
   See <<hbase.history,hbase.history>>.
+Why are the cells above 10MB not recommended for HBase?::
+  Large cells don't fit well into HBase's approach to buffering data. First, the large cells
bypass the MemStoreLAB when they are written. Then, they cannot be cached in the L2 block
cache during read operations. Instead, HBase has to allocate on-heap memory for them each
time. This can have a significant impact on the garbage collector within the RegionServer
 === Upgrading
 How do I upgrade Maven-managed projects from HBase 0.94 to HBase 0.96+?::
   In HBase 0.96, the project moved to a modular structure. Adjust your project's dependencies
to rely upon the `hbase-client` module or another module as appropriate, rather than a single
JAR. You can model your Maven dependency after one of the following, depending on your targeted
version of HBase. See Section 3.5, “Upgrading from 0.94.x to 0.96.x” or Section 3.3, “Upgrading
from 0.96.x to 0.98.x” for more information.

diff --git a/src/main/asciidoc/_chapters/hbase_mob.adoc b/src/main/asciidoc/_chapters/hbase_mob.adoc
index bdf077a..5da0343 100644
--- a/src/main/asciidoc/_chapters/hbase_mob.adoc
+++ b/src/main/asciidoc/_chapters/hbase_mob.adoc
@@ -36,7 +36,7 @@ read and write paths are optimized for values smaller than 100KB in size.
 HBase deals with large numbers of objects over this threshold, referred to here
 as medium objects, or MOBs, performance is degraded due to write amplification
 caused by splits and compactions. When using MOBs, ideally your objects will be between
-100KB and 10MB. HBase ***FIX_VERSION_NUMBER*** adds support
+100KB and 10MB (see the <<faq>>). HBase ***FIX_VERSION_NUMBER*** adds support
 for better managing large numbers of MOBs while maintaining performance,
 consistency, and low operational overhead. MOB support is provided by the work
 done in link:https://issues.apache.org/jira/browse/HBASE-11339[HBASE-11339]. To
@@ -155,7 +155,7 @@ family as the second argument. and take a compaction type as the third
 hbase> compact 't1', 'c1’, ‘MOB’
-hbase> major_compact_mob 't1', 'c1’, ‘MOB’
+hbase> major_compact 't1', 'c1’, ‘MOB’
 These commands are also available via `Admin.compact` and

diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc b/src/main/asciidoc/_chapters/ops_mgt.adoc
index e4c077f..b26e44b 100644
--- a/src/main/asciidoc/_chapters/ops_mgt.adoc
+++ b/src/main/asciidoc/_chapters/ops_mgt.adoc
@@ -1705,7 +1705,7 @@ handling multiple workloads:
 === Quotas
-HBASE-11598 introduces quotas, which allow you to throttle requests based on
+HBASE-11598 introduces RPC quotas, which allow you to throttle requests based on
 the following limits:
 . <<request-quotas,The number or size of requests(read, write, or read+write) in a
given timeframe>>
@@ -1885,6 +1885,84 @@ at the same time and that fewer scans can be executed at the same time.
A value
 `0.9` will give more queue/handlers to scans, so the number of scans executed will
 increase and the number of gets will decrease.
+=== Space Quotas
+link:https://issues.apache.org/jira/browse/HBASE-16961[HBASE-16961] introduces a new type
+quotas for HBase to leverage: filesystem quotas. These "space" quotas limit the amount of
+on the filesystem that HBase namespaces and tables can consume. If a user, malicious or ignorant,
+has the ability to write data into HBase, with enough time, that user can effectively crash
+(or worse HDFS) by consuming all available space. When there is no filesystem space available,
+HBase crashes because it can no longer create/sync data to the write-ahead log.
+This feature allows a for a limit to be set on the size of a table or namespace. When a space
quota is set
+on a namespace, the quota's limit applies to the sum of usage of all tables in that namespace.
+When a table with a quota exists in a namespace with a quota, the table quota takes priority
+over the namespace quota. This allows for a scenario where a large limit can be placed on
+a collection of tables, but a single table in that collection can have a fine-grained limit
+The existing `set_quota` and `list_quota` HBase shell commands can be used to interact with
+space quotas. Space quotas are quotas with a `TYPE` of `SPACE` and have `LIMIT` and `POLICY`
+attributes. The `LIMIT` is a string that refers to the amount of space on the filesystem
+that the quota subject (e.g. the table or namespace) may consume. For example, valid values
+of `LIMIT` are `'10G'`, `'2T'`, or `'256M'`. The `POLICY` refers to the action that HBase
+take when the quota subject's usage exceeds the `LIMIT`. The following are valid `POLICY`
+* `NO_INSERTS` - No new data may be written (e.g. `Put`, `Increment`, `Append`).
+* `NO_WRITES` - Same as `NO_INSERTS` but `Deletes` are also disallowed.
+* `NO_WRITES_COMPACTIONS` - Same as `NO_WRITES` but compactions are also disallowed.
+* `DISABLE` - The table(s) are disabled, preventing all read/write access.
+.Setting simple space quotas
+# Sets a quota on the table 't1' with a limit of 1GB, disallowing Puts/Increments/Appends
when the table exceeds 1GB
+hbase> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G', POLICY => NO_INSERTS
+# Sets a quota on the namespace 'ns1' with a limit of 50TB, disallowing Puts/Increments/Appends/Deletes
+hbase> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '50T', POLICY =>
+# Sets a quota on the table 't3' with a limit of 2TB, disallowing any writes and compactions
when the table exceeds 2TB.
+hbase> set_quota TYPE => SPACE, TABLE => 't3', LIMIT => '2T', POLICY => NO_WRITES_COMPACTIONS
+# Sets a quota on the table 't2' with a limit of 50GB, disabling the table when it exceeds
+hbase> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '50G', POLICY => DISABLE
+Consider the following scenario to set up quotas on a namespace, overriding the quota on
tables in that namespace
+.Table and Namespace space quotas
+hbase> create_namespace 'ns1'
+hbase> create 'ns1:t1'
+hbase> create 'ns1:t2'
+hbase> create 'ns1:t3'
+hbase> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '100T', POLICY =>
+hbase> set_quota TYPE => SPACE, TABLE => 'ns1:t2', LIMIT => '200G', POLICY =>
+hbase> set_quota TYPE => SPACE, TABLE => 'ns1:t3', LIMIT => '20T', POLICY =>
+In the above scenario, the tables in the namespace `ns1` will not be allowed to consume more
+100TB of space on the filesystem among each other. The table 'ns1:t2' is only allowed to
be 200GB in size, and will
+disallow all writes when the usage exceeds this limit. The table 'ns1:t3' is allowed to grow
to 20TB in size
+and also will disallow all writes then the usage exceeds this limit. Because there is no
table quota
+on 'ns1:t1', this table can grow up to 100TB, but only if 'ns1:t2' and 'ns1:t3' have a usage
of zero bytes.
+Practically, it's limit is 100TB less the current usage of 'ns1:t2' and 'ns1:t3'.
+=== Disabling Automatic Space Quota Deletion
+By default, if a table or namespace is deleted that has a space quota, the quota itself is
+also deleted. In some cases, it may be desirable for the space quota to not be automatically
+In these cases, the user may configure the system to not delete any space quota automatically
via hbase-site.xml.
+  <property>
+    <name>hbase.master.quota.observer.ignore</name>
+    <value>true</value>
+  </property>
 == HBase Backup

diff --git a/src/main/asciidoc/_chapters/orca.adoc b/src/main/asciidoc/_chapters/orca.adoc
index 1816b1a..0ba65ba 100644
--- a/src/main/asciidoc/_chapters/orca.adoc
+++ b/src/main/asciidoc/_chapters/orca.adoc
@@ -28,11 +28,15 @@
 :icons: font
-.Apache HBase Orca
+.Apache HBase Orca, HBase Colors, & Font
 link:https://issues.apache.org/jira/browse/HBASE-4920[An Orca is the Apache HBase mascot.]
See NOTICES.txt.
 Our Orca logo we got here: http://www.vectorfree.com/jumping-orca It is licensed Creative
Commons Attribution 3.0.
 See https://creativecommons.org/licenses/by/3.0/us/ We changed the logo by stripping the
colored background, inverting it and then rotating it some.
+The 'official' HBase color is "International Orange (Engineering)", the color of the link:https://en.wikipedia.org/wiki/International_orange[Golden
Gate bridge] in San Francisco and for space suits used by NASA.
+Our 'font' is link:http://www.dafont.com/bitsumishi.font[Bitsumishi].

diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc
index 7210040..9984cf5 100644
--- a/src/main/asciidoc/_chapters/upgrading.adoc
+++ b/src/main/asciidoc/_chapters/upgrading.adoc
@@ -127,7 +127,7 @@ for warning about incompatible changes). All effort will be made to provide
a de
 HBase has a lot of API points, but for the compatibility matrix above, we differentiate between
Client API, Limited Private API, and Private API. HBase uses a version of link:https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html[Hadoop's
Interface classification]. HBase's Interface classification classes can be found link:https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/classification/package-summary.html[here].
-* InterfaceAudience: captures the intended audience, possible values are Public (for end
users and external projects), LimitedPrivate (for other Projects, Coprocessors or other plugin
points), and Private (for internal use).
+* InterfaceAudience: captures the intended audience, possible values are Public (for end
users and external projects), LimitedPrivate (for other Projects, Coprocessors or other plugin
points), and Private (for internal use). Notice that, you may find that the classes which
are declared as IA.Private are used as parameter or return value for the interfaces which
are declared as IA.LimitedPrivate. This is possible. You should treat the IA.Private object
as a monolithic object, which means you can use it as a parameter to call other methods, or
return it, but you should never try to access its methods or fields.
 * InterfaceStability: describes what types of interface changes are permitted. Possible values
are Stable, Evolving, Unstable, and Deprecated. Notice that this annotation is only valid
for classes which are marked as IA.LimitedPrivate. The stability of IA.Public classes is only
related to the upgrade type(major, minor or patch). And for IA.Private classes, there is no
guarantee on the stability between releases. Refer to the Compatibility Matrix above for more

View raw message