hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bus...@apache.org
Subject [1/6] hbase git commit: HBASE-19223 Remove references to Date Tiered compaction from branch-1.2 ref guide
Date Wed, 15 Nov 2017 16:37:07 GMT
Repository: hbase
Updated Branches:
  refs/heads/branch-1.1 434097db0 -> 540bf082a
  refs/heads/branch-1.2 31677c0aa -> 041fbe71b
  refs/heads/branch-1.3 17f11ae6c -> 565527c60
  refs/heads/branch-1.4 846753c18 -> 9a075fe73
  refs/heads/branch-2 9c85d0017 -> fb79e9d4a
  refs/heads/master df98d6848 -> d89682ea9


HBASE-19223 Remove references to Date Tiered compaction from branch-1.2 ref guide

Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/041fbe71
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/041fbe71
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/041fbe71

Branch: refs/heads/branch-1.2
Commit: 041fbe71b6429170576395b35603e07157acc585
Parents: 31677c0
Author: Sean Busbey <busbey@apache.org>
Authored: Thu Nov 9 08:04:20 2017 -0600
Committer: Sean Busbey <busbey@apache.org>
Committed: Wed Nov 15 10:28:14 2017 -0600

----------------------------------------------------------------------
 src/main/asciidoc/_chapters/architecture.adoc | 101 ---------------------
 src/main/asciidoc/_chapters/upgrading.adoc    |  11 ++-
 2 files changed, 7 insertions(+), 105 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hbase/blob/041fbe71/src/main/asciidoc/_chapters/architecture.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/architecture.adoc b/src/main/asciidoc/_chapters/architecture.adoc
index 9f59cd5..6ab5f48 100644
--- a/src/main/asciidoc/_chapters/architecture.adoc
+++ b/src/main/asciidoc/_chapters/architecture.adoc
@@ -2060,107 +2060,6 @@ Why?
 
 NOTE: This information is now included in the configuration parameter table in <<compaction.parameters>>.
 
-[[ops.date.tiered]]
-===== Date Tiered Compaction
-
-Date tiered compaction is a date-aware store file compaction strategy that is beneficial
for time-range scans for time-series data.
-
-[[ops.date.tiered.when]]
-===== When To Use Date Tiered Compactions
-
-Consider using Date Tiered Compaction for reads for limited time ranges, especially scans
of recent data
-
-Don't use it for
-
-* random gets without a limited time range
-* frequent deletes and updates
-* Frequent out of order data writes creating long tails, especially writes with future timestamps
-* frequent bulk loads with heavily overlapping time ranges
-
-.Performance Improvements
-Performance testing has shown that the performance of time-range scans improve greatly for
limited time ranges, especially scans of recent data.
-
-[[ops.date.tiered.enable]]
-====== Enabling Date Tiered Compaction
-
-You can enable Date Tiered compaction for a table or a column family, by setting its `hbase.hstore.engine.class`
to `org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine`.
-
-You also need to set `hbase.hstore.blockingStoreFiles` to a high number, such as 60, if using
all default settings, rather than the default value of 12). Use 1.5~2 x projected file count
if changing the parameters, Projected file count = windows per tier x tier count + incoming
window min + files older than max age
-
-You also need to set `hbase.hstore.compaction.max` to the same value as `hbase.hstore.blockingStoreFiles`
to unblock major compaction.
-
-.Procedure: Enable Date Tiered Compaction
-. Run one of following commands in the HBase shell.
-  Replace the table name `orders_table` with the name of your table.
-+
-[source,sql]
-----
-alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine',
'hbase.hstore.blockingStoreFiles' => '60', 'hbase.hstore.compaction.min'=>'2', 'hbase.hstore.compaction.max'=>'60'}
-alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class'
=> 'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine', 'hbase.hstore.blockingStoreFiles'
=> '60', 'hbase.hstore.compaction.min'=>'2', 'hbase.hstore.compaction.max'=>'60'}}
-create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' =>
'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine', 'hbase.hstore.blockingStoreFiles'
=> '60', 'hbase.hstore.compaction.min'=>'2', 'hbase.hstore.compaction.max'=>'60'}
-----
-
-. Configure other options if needed.
-  See <<ops.date.tiered.config>> for more information.
-
-.Procedure: Disable Date Tiered Compaction
-. Set the `hbase.hstore.engine.class` option to either nil or `org.apache.hadoop.hbase.regionserver.DefaultStoreEngine`.
-  Either option has the same effect.
-  Make sure you set the other options you changed to the original settings too.
-+
-[source,sql]
-----
-alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.DefaultStoreEngine',
'hbase.hstore.blockingStoreFiles' => '12', 'hbase.hstore.compaction.min'=>'6', 'hbase.hstore.compaction.max'=>'12'}}
-----
-
-When you change the store engine either way, a major compaction will likely be performed
on most regions.
-This is not necessary on new tables.
-
-[[ops.date.tiered.config]]
-====== Configuring Date Tiered Compaction
-
-Each of the settings for date tiered compaction should be configured at the table or column
family level.
-If you use HBase shell, the general command pattern is as follows:
-
-[source,sql]
-----
-alter 'orders_table', CONFIGURATION => {'key' => 'value', ..., 'key' => 'value'}}
-----
-
-[[ops.date.tiered.config.parameters]]
-.Tier Parameters
-
-You can configure your date tiers by changing the settings for the following parameters:
-
-.Date Tier Parameters
-[cols="1,1a", frame="all", options="header"]
-|===
-| Setting
-| Notes
-
-|`hbase.hstore.compaction.date.tiered.max.storefile.age.millis`
-|Files with max-timestamp smaller than this will no longer be compacted.Default at Long.MAX_VALUE.
-
-| `hbase.hstore.compaction.date.tiered.base.window.millis`
-| Base window size in milliseconds. Default at 6 hours.
-
-| `hbase.hstore.compaction.date.tiered.windows.per.tier`
-| Number of windows per tier. Default at 4.
-
-| `hbase.hstore.compaction.date.tiered.incoming.window.min`
-| Minimal number of files to compact in the incoming window. Set it to expected number of
files in the window to avoid wasteful compaction. Default at 6.
-
-| `hbase.hstore.compaction.date.tiered.window.policy.class`
-| The policy to select store files within the same time window. It doesn’t apply to the
incoming window. Default at exploring compaction. This is to avoid wasteful compaction.
-|===
-
-[[ops.date.tiered.config.compaction.throttler]]
-.Compaction Throttler
-
-With tiered compaction all servers in the cluster will promote windows to higher tier at
the same time, so using a compaction throttle is recommended:
-Set `hbase.regionserver.throughput.controller` to `org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController`.
-
-NOTE: For more information about date tiered compaction, please refer to the design specification
at https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8
 [[ops.stripe]]
 ===== Experimental: Stripe Compactions
 

http://git-wip-us.apache.org/repos/asf/hbase/blob/041fbe71/src/main/asciidoc/_chapters/upgrading.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc
index d731542..3acb36e 100644
--- a/src/main/asciidoc/_chapters/upgrading.adoc
+++ b/src/main/asciidoc/_chapters/upgrading.adoc
@@ -176,16 +176,16 @@ In the minor version-particular sections below, we call out where the
versions a
 == Upgrade Paths
 
 [[upgrade1.0]]
-=== Upgrading from 0.98.x to 1.0.x
+=== Upgrading from 0.98.x to 1.x
 
-In this section we first note the significant changes that come in with 1.0.0 HBase and then
we go over the upgrade process. Be sure to read the significant changes section with care
so you avoid surprises.
+In this section we first note the significant changes that come in with 1.0.0+ HBase and
then we go over the upgrade process. Be sure to read the significant changes section with
care so you avoid surprises.
 
 ==== Changes of Note!
 
-In here we list important changes that are in 1.0.0 since 0.98.x., changes you should be
aware that will go into effect once you upgrade.
+In here we list important changes that are in 1.0.0+ since 0.98.x., changes you should be
aware that will go into effect once you upgrade.
 
 [[zookeeper.3.4]]
-.ZooKeeper 3.4 is required in HBase 1.0.0
+.ZooKeeper 3.4 is required in HBase 1.0.0+
 See <<zookeeper.requirements>>.
 
 [[default.ports.changed]]
@@ -218,6 +218,9 @@ to miss data. In particular, 0.98.11 defaults `hbase.client.scanner.max.result.s
 to 2 MB but other versions default to larger values. For this reason, be very careful
 using 0.98.11 servers with any other client version.
 
+.Availability of Date Tiered Compaction.
+The Date Tiered Compaction feature available as of 0.98.19 is available in the 1.y release
line starting in release 1.3.0. If you have enabled this feature for any tables you must upgrade
to version 1.3.0 or later. If you attempt to use an earlier 1.y release, any tables configured
to use date tiered compaction will fail to have their regions open.
+
 [[upgrade1.0.rolling.upgrade]]
 ==== Rolling upgrade from 0.98.x to HBase 1.0.0
 .From 0.96.x to 1.0.0


Mime
View raw message