hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mberto...@apache.org
Subject [2/2] git commit: HBASE-11737 Document callQueue improvements from HBASE-11355 and HBASE-11724 (Misty Stanley-Jones)
Date Wed, 20 Aug 2014 08:45:24 GMT
HBASE-11737 Document callQueue improvements from HBASE-11355 and HBASE-11724 (Misty Stanley-Jones)


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/5c1ae840
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/5c1ae840
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/5c1ae840

Branch: refs/heads/branch-1
Commit: 5c1ae840f21f7a3857543e408ef20a63be2b0751
Parents: cbc1bc7
Author: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
Authored: Wed Aug 20 09:40:58 2014 +0100
Committer: Matteo Bertozzi <matteo.bertozzi@cloudera.com>
Committed: Wed Aug 20 09:42:41 2014 +0100

----------------------------------------------------------------------
 src/main/docbkx/performance.xml | 126 ++++++++++++++++++++++++++++++++++-
 1 file changed, 125 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hbase/blob/5c1ae840/src/main/docbkx/performance.xml
----------------------------------------------------------------------
diff --git a/src/main/docbkx/performance.xml b/src/main/docbkx/performance.xml
index 187d8c2..58557db 100644
--- a/src/main/docbkx/performance.xml
+++ b/src/main/docbkx/performance.xml
@@ -252,7 +252,131 @@
         release note on <link xlink:href="https://issues.apache.org/jira/browse/HBASE-5074">HBASE-5074
support checksums in HBase block cache</link>.
         </para>
     </section>
-
+    <section>
+      <title>Tuning <code>callQueue</code> Options</title>
+      <para><link xlink:href="https://issues.apache.org/jira/browse/HBASE-11355">HBASE-11355</link>
+        introduces several callQueue tuning mechanisms which can increase performance. See
the JIRA
+        for some benchmarking information.</para>
+      <itemizedlist>
+        <listitem>
+          <para>To increase the number of callqueues, set
+              <option>hbase.ipc.server.num.callqueue</option> to a value greater
than
+              <literal>1</literal>.</para>
+        </listitem>
+        <listitem>
+          <para>To split the callqueue into separate read and write queues, set
+              <code>hbase.ipc.server.callqueue.read.ratio</code> to a value between
+              <literal>0</literal> and <literal>1</literal>. This
factor weights the queues toward
+            writes (if below .5) or reads (if above .5). Another way to say this is that
the factor
+            determines what percentage of the split queues are used for reads. The following
+            examples illustrate some of the possibilities. Note that you always have at least
one
+            write queue, no matter what setting you use.</para>
+          <itemizedlist>
+            <listitem>
+              <para>The default value of <literal>0</literal> does not
split the queue.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>.3</literal> uses 30% of the queues
for reading and 60% for
+                writing. Given a value of <literal>10</literal> for
+                  <option>hbase.ipc.server.num.callqueue</option>, 3 queues would
be used for reads
+                and 7 for writes.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>.5</literal> uses the same number
of read queues and write
+                queues. Given a value of <literal>10</literal> for
+                <option>hbase.ipc.server.num.callqueue</option>, 5 queues would
be used for reads
+                and 5 for writes.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>.6</literal> uses 60% of the queues
for reading and 30% for
+                reading. Given a value of <literal>10</literal> for
+                <option>hbase.ipc.server.num.callqueue</option>, 7 queues would
be used for reads
+                and 3 for writes.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>1.0</literal> uses one queue to
process write requests, and
+                all other queues process read requests. A value higher than <literal>1.0</literal>
+                has the same effect as a value of <literal>1.0</literal>. Given
a value of
+                  <literal>10</literal> for <option>hbase.ipc.server.num.callqueue</option>,
9
+                queues would be used for reads and 1 for writes.</para>
+            </listitem>
+          </itemizedlist>
+        </listitem>
+        <listitem>
+          <para>You can also split the read queues so that separate queues are used
for short reads
+            (from Get operations) and long reads (from Scan operations), by setting the
+              <option>hbase.ipc.server.callqueue.scan.ratio</option> option.
This option is a factor
+            between 0 and 1, which determine the ratio of read queues used for Gets and Scans.
More
+            queues are used for Gets if the value is below <literal>.5</literal>
and more are used
+            for scans if the value is above <literal>.5</literal>. No matter
what setting you use,
+            at least one read queue is used for Get operations.</para>
+          <itemizedlist>
+            <listitem>
+              <para>A value of <literal>0</literal> does not split the
read queue.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>.3</literal> uses 60% of the read
queues for Gets and 30%
+                for Scans. Given a value of <literal>20</literal> for
+                  <option>hbase.ipc.server.num.callqueue</option> and a value
of <literal>.5
+                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>,
10 queues
+                would be used for reads, out of those 10, 7 would be used for Gets and 3
for
+                Scans.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>.5</literal> uses half the read
queues for Gets and half for
+                Scans. Given a value of <literal>20</literal> for
+                  <option>hbase.ipc.server.num.callqueue</option> and a value
of <literal>.5
+                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>,
10 queues
+                would be used for reads, out of those 10, 5 would be used for Gets and 5
for
+                Scans.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>.6</literal> uses 30% of the read
queues for Gets and 60%
+                for Scans. Given a value of <literal>20</literal> for
+                  <option>hbase.ipc.server.num.callqueue</option> and a value
of <literal>.5
+                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>,
10 queues
+                would be used for reads, out of those 10, 3 would be used for Gets and 7
for
+                Scans.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>1.0</literal> uses all but one
of the read queues for Scans.
+                Given a value of <literal>20</literal> for
+                  <option>hbase.ipc.server.num.callqueue</option> and a value
of <literal>.5
+                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>,
10 queues
+                would be used for reads, out of those 10, 1 would be used for Gets and 9
for
+                Scans.</para>
+            </listitem>
+          </itemizedlist>
+        </listitem>
+        <listitem>
+          <para>You can use the new option
+              <option>hbase.ipc.server.callqueue.handler.factor</option> to programmatically
tune
+            the number of queues:</para>
+          <itemizedlist>
+            <listitem>
+              <para>A value of <literal>0</literal> uses a single shared
queue between all the
+                handlers.</para>
+            </listitem>
+            <listitem>
+              <para>A value of <literal>1</literal> uses a separate queue
for each handler.</para>
+            </listitem>
+            <listitem>
+              <para>A value between <literal>0</literal> and <literal>1</literal>
tunes the number
+                of queues against the number of handlers. For instance, a value of
+                  <literal>.5</literal> shares one queue between each two handlers.</para>
+            </listitem>
+          </itemizedlist>
+          <para>Having more queues, such as in a situation where you have one queue
per handler,
+            reduces contention when adding a task to a queue or selecting it from a queue.
The
+            trade-off is that if you have some queues with long-running tasks, a handler
may end up
+            waiting to execute from that queue rather than processing another queue which
has
+            waiting tasks.</para>
+        </listitem>
+      </itemizedlist>
+      <para>For these values to take effect on a given Region Server, the Region Server
must be
+        restarted. These parameters are intended for testing purposes and should be used
+        carefully.</para>
+    </section>
   </section>
 
 


Mime
View raw message