hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mi...@apache.org
Subject hbase git commit: HBASE-11339 Addendum: Put back the sweeper tool docs for now
Date Sun, 08 Mar 2015 23:17:20 GMT
Repository: hbase
Updated Branches:
  refs/heads/hbase-11339 8b4671c10 -> 33a6a819a


HBASE-11339 Addendum: Put back the sweeper tool docs for now


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/33a6a819
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/33a6a819
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/33a6a819

Branch: refs/heads/hbase-11339
Commit: 33a6a819a467e09ce80e7d42362c774e62d35809
Parents: 8b4671c
Author: Misty Stanley-Jones <mstanleyjones@cloudera.com>
Authored: Mon Mar 9 09:16:59 2015 +1000
Committer: Misty Stanley-Jones <mstanleyjones@cloudera.com>
Committed: Mon Mar 9 09:17:06 2015 +1000

----------------------------------------------------------------------
 src/main/asciidoc/_chapters/hbase_mob.adoc | 64 +++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hbase/blob/33a6a819/src/main/asciidoc/_chapters/hbase_mob.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/hbase_mob.adoc b/src/main/asciidoc/_chapters/hbase_mob.adoc
index 8dea211..0882de8 100644
--- a/src/main/asciidoc/_chapters/hbase_mob.adoc
+++ b/src/main/asciidoc/_chapters/hbase_mob.adoc
@@ -118,3 +118,67 @@ Because there can be a large number of MOB files at any time, as compared
to the
 ----
 ====
 
+=== MOB Optimization Tasks
+
+HBase MOB currently relies on a MapReduce job called the Sweeper tool for optimization. The
Sweeper tool oalesces small MOB files or MOB files with many deletions or updates. A native
MOB compaction tool is still in testing. To configure the Sweeper tool, set the following
options:
+
+[source,xml]
+----
+<property>
+    <name>hbase.mob.sweep.tool.compaction.ratio</name>
+    <value>0.5f</value>
+    <description>
+      If there are too many cells deleted in a mob file, it's regarded
+      as an invalid file and needs to be merged.
+      If existingCellsSize/mobFileSize is less than ratio, it's regarded
+      as an invalid file. The default value is 0.5f.
+    </description>
+</property>
+<property>
+    <name>hbase.mob.sweep.tool.compaction.mergeable.size</name>
+    <value>134217728</value>
+    <description>
+      If the size of a mob file is less than this value, it's regarded as a small
+      file and needs to be merged. The default value is 128MB.
+    </description>
+</property>
+<property>
+    <name>hbase.mob.sweep.tool.compaction.memstore.flush.size</name>
+    <value>134217728</value>
+    <description>
+      The flush size for the memstore used by sweep job. Each sweep reducer owns such a memstore.
+      The default value is 128MB.
+    </description>
+</property>
+<property>
+    <name>hbase.mob.cleaner.interval</name>
+    <value>86400000</value>
+    <description>
+      The period that ExpiredMobFileCleaner runs. The unit is millisecond.
+      The default value is one day.
+    </description>
+</property>
+----
+
+Next, add the HBase install directory, _`$HBASE_HOME`/*_, and HBase library directory to
_yarn-site.xml_ Adjust this example to suit your environment.
+[source,xml]
+----
+<property>
+    <description>Classpath for typical applications.</description>
+    <name>yarn.application.classpath</name>
+    <value>
+        $HADOOP_CONF_DIR,
+        $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
+        $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
+        $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
+        $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*,
+        $HBASE_HOME/*, $HBASE_HOME/lib/*
+    </value>
+</property>
+----
+
+Finally, run the `sweeper` tool for each column which is configured for MOB.
+[source,bash]
+----
+$ org.apache.hadoop.hbase.mob.compactions.Sweeper _tableName_ _familyName_
+----


Mime
View raw message