hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bus...@apache.org
Subject [07/15] hbase git commit: HBASE-14066 clean out old docbook docs from branch-1.
Date Tue, 14 Jul 2015 02:50:23 GMT
http://git-wip-us.apache.org/repos/asf/hbase/blob/09a422e8/src/main/docbkx/performance.xml
----------------------------------------------------------------------
diff --git a/src/main/docbkx/performance.xml b/src/main/docbkx/performance.xml
deleted file mode 100644
index 1757d3f..0000000
--- a/src/main/docbkx/performance.xml
+++ /dev/null
@@ -1,1207 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<chapter
-  version="5.0"
-  xml:id="performance"
-  xmlns="http://docbook.org/ns/docbook"
-  xmlns:xlink="http://www.w3.org/1999/xlink"
-  xmlns:xi="http://www.w3.org/2001/XInclude"
-  xmlns:svg="http://www.w3.org/2000/svg"
-  xmlns:m="http://www.w3.org/1998/Math/MathML"
-  xmlns:html="http://www.w3.org/1999/xhtml"
-  xmlns:db="http://docbook.org/ns/docbook">
-  <!--
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *     http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
--->
-  <title>Apache HBase Performance Tuning</title>
-
-  <section
-    xml:id="perf.os">
-    <title>Operating System</title>
-    <section
-      xml:id="perf.os.ram">
-      <title>Memory</title>
-      <para>RAM, RAM, RAM. Don't starve HBase.</para>
-    </section>
-    <section
-      xml:id="perf.os.64">
-      <title>64-bit</title>
-      <para>Use a 64-bit platform (and 64-bit JVM).</para>
-    </section>
-    <section
-      xml:id="perf.os.swap">
-      <title>Swapping</title>
-      <para>Watch out for swapping. Set swappiness to 0.</para>
-    </section>
-  </section>
-  <section
-    xml:id="perf.network">
-    <title>Network</title>
-    <para> Perhaps the most important factor in avoiding network issues degrading Hadoop and HBase
-      performance is the switching hardware that is used, decisions made early in the scope of the
-      project can cause major problems when you double or triple the size of your cluster (or more). </para>
-    <para> Important items to consider: <itemizedlist>
-        <listitem>
-          <para>Switching capacity of the device</para>
-        </listitem>
-        <listitem>
-          <para>Number of systems connected</para>
-        </listitem>
-        <listitem>
-          <para>Uplink capacity</para>
-        </listitem>
-      </itemizedlist>
-    </para>
-    <section
-      xml:id="perf.network.1switch">
-      <title>Single Switch</title>
-      <para>The single most important factor in this configuration is that the switching capacity of
-        the hardware is capable of handling the traffic which can be generated by all systems
-        connected to the switch. Some lower priced commodity hardware can have a slower switching
-        capacity than could be utilized by a full switch. </para>
-    </section>
-    <section
-      xml:id="perf.network.2switch">
-      <title>Multiple Switches</title>
-      <para>Multiple switches are a potential pitfall in the architecture. The most common
-        configuration of lower priced hardware is a simple 1Gbps uplink from one switch to another.
-        This often overlooked pinch point can easily become a bottleneck for cluster communication.
-        Especially with MapReduce jobs that are both reading and writing a lot of data the
-        communication across this uplink could be saturated. </para>
-      <para>Mitigation of this issue is fairly simple and can be accomplished in multiple ways: </para>
-      <itemizedlist>
-        <listitem>
-          <para>Use appropriate hardware for the scale of the cluster which you're attempting to
-            build.</para>
-        </listitem>
-        <listitem>
-          <para>Use larger single switch configurations i.e. single 48 port as opposed to 2x 24
-            port</para>
-        </listitem>
-        <listitem>
-          <para>Configure port trunking for uplinks to utilize multiple interfaces to increase cross
-            switch bandwidth.</para>
-        </listitem>
-      </itemizedlist>
-    </section>
-    <section
-      xml:id="perf.network.multirack">
-      <title>Multiple Racks</title>
-      <para>Multiple rack configurations carry the same potential issues as multiple switches, and
-        can suffer performance degradation from two main areas: </para>
-      <itemizedlist>
-        <listitem>
-          <para>Poor switch capacity performance</para>
-        </listitem>
-        <listitem>
-          <para>Insufficient uplink to another rack</para>
-        </listitem>
-      </itemizedlist>
-      <para>If the the switches in your rack have appropriate switching capacity to handle all the
-        hosts at full speed, the next most likely issue will be caused by homing more of your
-        cluster across racks. The easiest way to avoid issues when spanning multiple racks is to use
-        port trunking to create a bonded uplink to other racks. The downside of this method however,
-        is in the overhead of ports that could potentially be used. An example of this is, creating
-        an 8Gbps port channel from rack A to rack B, using 8 of your 24 ports to communicate between
-        racks gives you a poor ROI, using too few however can mean you're not getting the most out
-        of your cluster. </para>
-      <para>Using 10Gbe links between racks will greatly increase performance, and assuming your
-        switches support a 10Gbe uplink or allow for an expansion card will allow you to save your
-        ports for machines as opposed to uplinks. </para>
-    </section>
-    <section
-      xml:id="perf.network.ints">
-      <title>Network Interfaces</title>
-      <para>Are all the network interfaces functioning correctly? Are you sure? See the
-        Troubleshooting Case Study in <xref
-          linkend="casestudies.slownode" />. </para>
-    </section>
-  </section>
-  <!-- network -->
-
-  <section
-    xml:id="jvm">
-    <title>Java</title>
-
-    <section
-      xml:id="gc">
-      <title>The Garbage Collector and Apache HBase</title>
-
-      <section
-        xml:id="gcpause">
-        <title>Long GC pauses</title>
-
-        <para xml:id="mslab">In his presentation, <link
-            xlink:href="http://www.slideshare.net/cloudera/hbase-hug-presentation">Avoiding Full GCs
-            with MemStore-Local Allocation Buffers</link>, Todd Lipcon describes two cases of
-          stop-the-world garbage collections common in HBase, especially during loading; CMS failure
-          modes and old generation heap fragmentation brought. To address the first, start the CMS
-          earlier than default by adding <code>-XX:CMSInitiatingOccupancyFraction</code> and setting
-          it down from defaults. Start at 60 or 70 percent (The lower you bring down the threshold,
-          the more GCing is done, the more CPU used). To address the second fragmentation issue,
-          Todd added an experimental facility, <indexterm><primary>MSLAB</primary></indexterm>, that
-          must be explicitly enabled in Apache HBase 0.90.x (Its defaulted to be on in Apache 0.92.x
-          HBase). See <code>hbase.hregion.memstore.mslab.enabled</code> to true in your
-            <classname>Configuration</classname>. See the cited slides for background and detail.
-          The latest jvms do better regards fragmentation so make sure you are running a recent
-          release. Read down in the message, <link
-            xlink:href="http://osdir.com/ml/hotspot-gc-use/2011-11/msg00002.html">Identifying
-            concurrent mode failures caused by fragmentation</link>. Be aware that when enabled,
-          each MemStore instance will occupy at least an MSLAB instance of memory. If you have
-          thousands of regions or lots of regions each with many column families, this allocation of
-          MSLAB may be responsible for a good portion of your heap allocation and in an extreme case
-          cause you to OOME. Disable MSLAB in this case, or lower the amount of memory it uses or
-          float less regions per server. </para>
-        <para>If you have a write-heavy workload, check out <link
-            xlink:href="https://issues.apache.org/jira/browse/HBASE-8163">HBASE-8163
-            MemStoreChunkPool: An improvement for JAVA GC when using MSLAB</link>. It describes
-          configurations to lower the amount of young GC during write-heavy loadings. If you do not
-          have HBASE-8163 installed, and you are trying to improve your young GC times, one trick to
-          consider -- courtesy of our Liang Xie -- is to set the GC config
-            <varname>-XX:PretenureSizeThreshold</varname> in <filename>hbase-env.sh</filename> to be
-          just smaller than the size of <varname>hbase.hregion.memstore.mslab.chunksize</varname> so
-          MSLAB allocations happen in the tenured space directly rather than first in the young gen.
-          You'd do this because these MSLAB allocations are going to likely make it to the old gen
-          anyways and rather than pay the price of a copies between s0 and s1 in eden space followed
-          by the copy up from young to old gen after the MSLABs have achieved sufficient tenure,
-          save a bit of YGC churn and allocate in the old gen directly. </para>
-        <para>For more information about GC logs, see <xref
-            linkend="trouble.log.gc" />. </para>
-    <para>Consider also enabling the offheap Block Cache.  This has been shown to mitigate
-        GC pause times.  See <xref linkend="block.cache" /></para>
-      </section>
-    </section>
-  </section>
-
-  <section
-    xml:id="perf.configurations">
-    <title>HBase Configurations</title>
-
-    <para>See <xref
-        linkend="recommended_configurations" />.</para>
-
-    <section
-      xml:id="perf.compactions.and.splits">
-      <title>Managing Compactions</title>
-
-      <para>For larger systems, managing <link
-      linkend="disable.splitting">compactions and splits</link> may be
-      something you want to consider.</para>
-    </section>
-
-    <section xml:id="perf.handlers">
-        <title><varname>hbase.regionserver.handler.count</varname></title>
-        <para>See <xref linkend="hbase.regionserver.handler.count"/>.
-	    </para>
-    </section>
-    
-
-
-    <section xml:id="perf.hfile.block.cache.size">
-        <title><varname>hfile.block.cache.size</varname></title>
-        <para>See <xref linkend="hfile.block.cache.size"/>.
-        A memory setting for the RegionServer process.
-        </para>
-    </section>
-    <section xml:id="blockcache.prefetch">
-      <title>Prefetch Option for Blockcache</title>
-      <para><link xlink:href="https://issues.apache.org/jira/browse/HBASE-9857">HBASE-9857</link>
-        adds a new option to prefetch HFile contents when opening the blockcache, if a columnfamily
-        or regionserver property is set. This option is available for HBase 0.98.3 and later. The
-        purpose is to warm the blockcache as rapidly as possible after the cache is opened, using
-        in-memory table data, and not counting the prefetching as cache misses. This is great for
-        fast reads, but is not a good idea if the data to be preloaded will not fit into the
-        blockcache. It is useful for tuning the IO impact of prefetching versus the time before all
-        data blocks are in cache. </para>
-      <para>To enable prefetching on a given column family, you can use HBase Shell or use the
-        API.</para>
-      <example>
-        <title>Enable Prefetch Using HBase Shell</title>
-        <screen>hbase> create 'MyTable', { NAME => 'myCF', PREFETCH_BLOCKS_ON_OPEN => 'true' }</screen>
-      </example>
-      <example>
-        <title>Enable Prefetch Using the API</title>
-        <programlisting language="java">
-// ...
-HTableDescriptor tableDesc = new HTableDescriptor("myTable");
-HColumnDescriptor cfDesc = new HColumnDescriptor("myCF");
-cfDesc.setPrefetchBlocksOnOpen(true);
-tableDesc.addFamily(cfDesc);
-// ...        
-        </programlisting>
-      </example>
-      <para>See the API documentation for <link
-          xlink:href="https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/io/hfile/CacheConfig.html"
-          >CacheConfig</link>.</para>
-    </section>
-    <section xml:id="perf.rs.memstore.size">
-        <title><varname>hbase.regionserver.global.memstore.size</varname></title>
-        <para>See <xref linkend="hbase.regionserver.global.memstore.size"/>.
-        This memory setting is often adjusted for the RegionServer process depending on needs.
-        </para>
-    </section>
-    <section xml:id="perf.rs.memstore.size.lower.limit">
-        <title><varname>hbase.regionserver.global.memstore.size.lower.limit</varname></title>
-        <para>See <xref linkend="hbase.regionserver.global.memstore.size.lower.limit"/>.
-        This memory setting is often adjusted for the RegionServer process depending on needs.
-        </para>
-    </section>
-    <section xml:id="perf.hstore.blockingstorefiles">
-        <title><varname>hbase.hstore.blockingStoreFiles</varname></title>
-        <para>See <xref linkend="hbase.hstore.blockingStoreFiles"/>.
-        If there is blocking in the RegionServer logs, increasing this can help.
-        </para>
-    </section>
-    <section xml:id="perf.hregion.memstore.block.multiplier">
-        <title><varname>hbase.hregion.memstore.block.multiplier</varname></title>
-        <para>See <xref linkend="hbase.hregion.memstore.block.multiplier"/>.
-        If there is enough RAM, increasing this can help.
-        </para>
-    </section>
-    <section xml:id="hbase.regionserver.checksum.verify">
-        <title><varname>hbase.regionserver.checksum.verify</varname></title>
-        <para>Have HBase write the checksum into the datablock and save
-        having to do the checksum seek whenever you read.</para>
-
-        <para>See <xref linkend="hbase.regionserver.checksum.verify"/>,
-        <xref linkend="hbase.hstore.bytes.per.checksum"/> and <xref linkend="hbase.hstore.checksum.algorithm"/>
-        For more information see the
-        release note on <link xlink:href="https://issues.apache.org/jira/browse/HBASE-5074">HBASE-5074 support checksums in HBase block cache</link>.
-        </para>
-    </section>
-    <section>
-      <title>Tuning <code>callQueue</code> Options</title>
-      <para><link xlink:href="https://issues.apache.org/jira/browse/HBASE-11355">HBASE-11355</link>
-        introduces several callQueue tuning mechanisms which can increase performance. See the JIRA
-        for some benchmarking information.</para>
-      <itemizedlist>
-        <listitem>
-          <para>To increase the number of callqueues, set
-              <option>hbase.ipc.server.num.callqueue</option> to a value greater than
-              <literal>1</literal>.</para>
-        </listitem>
-        <listitem>
-          <para>To split the callqueue into separate read and write queues, set
-              <code>hbase.ipc.server.callqueue.read.ratio</code> to a value between
-              <literal>0</literal> and <literal>1</literal>. This factor weights the queues toward
-            writes (if below .5) or reads (if above .5). Another way to say this is that the factor
-            determines what percentage of the split queues are used for reads. The following
-            examples illustrate some of the possibilities. Note that you always have at least one
-            write queue, no matter what setting you use.</para>
-          <itemizedlist>
-            <listitem>
-              <para>The default value of <literal>0</literal> does not split the queue.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>.3</literal> uses 30% of the queues for reading and 60% for
-                writing. Given a value of <literal>10</literal> for
-                  <option>hbase.ipc.server.num.callqueue</option>, 3 queues would be used for reads
-                and 7 for writes.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>.5</literal> uses the same number of read queues and write
-                queues. Given a value of <literal>10</literal> for
-                <option>hbase.ipc.server.num.callqueue</option>, 5 queues would be used for reads
-                and 5 for writes.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>.6</literal> uses 60% of the queues for reading and 30% for
-                reading. Given a value of <literal>10</literal> for
-                <option>hbase.ipc.server.num.callqueue</option>, 7 queues would be used for reads
-                and 3 for writes.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>1.0</literal> uses one queue to process write requests, and
-                all other queues process read requests. A value higher than <literal>1.0</literal>
-                has the same effect as a value of <literal>1.0</literal>. Given a value of
-                  <literal>10</literal> for <option>hbase.ipc.server.num.callqueue</option>, 9
-                queues would be used for reads and 1 for writes.</para>
-            </listitem>
-          </itemizedlist>
-        </listitem>
-        <listitem>
-          <para>You can also split the read queues so that separate queues are used for short reads
-            (from Get operations) and long reads (from Scan operations), by setting the
-              <option>hbase.ipc.server.callqueue.scan.ratio</option> option. This option is a factor
-            between 0 and 1, which determine the ratio of read queues used for Gets and Scans. More
-            queues are used for Gets if the value is below <literal>.5</literal> and more are used
-            for scans if the value is above <literal>.5</literal>. No matter what setting you use,
-            at least one read queue is used for Get operations.</para>
-          <itemizedlist>
-            <listitem>
-              <para>A value of <literal>0</literal> does not split the read queue.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>.3</literal> uses 60% of the read queues for Gets and 30%
-                for Scans. Given a value of <literal>20</literal> for
-                  <option>hbase.ipc.server.num.callqueue</option> and a value of <literal>.5
-                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>, 10 queues
-                would be used for reads, out of those 10, 7 would be used for Gets and 3 for
-                Scans.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>.5</literal> uses half the read queues for Gets and half for
-                Scans. Given a value of <literal>20</literal> for
-                  <option>hbase.ipc.server.num.callqueue</option> and a value of <literal>.5
-                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>, 10 queues
-                would be used for reads, out of those 10, 5 would be used for Gets and 5 for
-                Scans.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>.6</literal> uses 30% of the read queues for Gets and 60%
-                for Scans. Given a value of <literal>20</literal> for
-                  <option>hbase.ipc.server.num.callqueue</option> and a value of <literal>.5
-                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>, 10 queues
-                would be used for reads, out of those 10, 3 would be used for Gets and 7 for
-                Scans.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>1.0</literal> uses all but one of the read queues for Scans.
-                Given a value of <literal>20</literal> for
-                  <option>hbase.ipc.server.num.callqueue</option> and a value of <literal>.5
-                </literal> for <option>hbase.ipc.server.callqueue.read.ratio</option>, 10 queues
-                would be used for reads, out of those 10, 1 would be used for Gets and 9 for
-                Scans.</para>
-            </listitem>
-          </itemizedlist>
-        </listitem>
-        <listitem>
-          <para>You can use the new option
-              <option>hbase.ipc.server.callqueue.handler.factor</option> to programmatically tune
-            the number of queues:</para>
-          <itemizedlist>
-            <listitem>
-              <para>A value of <literal>0</literal> uses a single shared queue between all the
-                handlers.</para>
-            </listitem>
-            <listitem>
-              <para>A value of <literal>1</literal> uses a separate queue for each handler.</para>
-            </listitem>
-            <listitem>
-              <para>A value between <literal>0</literal> and <literal>1</literal> tunes the number
-                of queues against the number of handlers. For instance, a value of
-                  <literal>.5</literal> shares one queue between each two handlers.</para>
-            </listitem>
-          </itemizedlist>
-          <para>Having more queues, such as in a situation where you have one queue per handler,
-            reduces contention when adding a task to a queue or selecting it from a queue. The
-            trade-off is that if you have some queues with long-running tasks, a handler may end up
-            waiting to execute from that queue rather than processing another queue which has
-            waiting tasks.</para>
-        </listitem>
-      </itemizedlist>
-      <para>For these values to take effect on a given Region Server, the Region Server must be
-        restarted. These parameters are intended for testing purposes and should be used
-        carefully.</para>
-    </section>
-  </section>
-
-
-
-
-  <section
-    xml:id="perf.zookeeper">
-    <title>ZooKeeper</title>
-    <para>See <xref
-        linkend="zookeeper" /> for information on configuring ZooKeeper, and see the part about
-      having a dedicated disk. </para>
-  </section>
-  <section
-    xml:id="perf.schema">
-    <title>Schema Design</title>
-
-    <section
-      xml:id="perf.number.of.cfs">
-      <title>Number of Column Families</title>
-      <para>See <xref
-          linkend="number.of.cfs" />.</para>
-    </section>
-    <section
-      xml:id="perf.schema.keys">
-      <title>Key and Attribute Lengths</title>
-      <para>See <xref
-          linkend="keysize" />. See also <xref
-          linkend="perf.compression.however" /> for compression caveats.</para>
-    </section>
-    <section
-      xml:id="schema.regionsize">
-      <title>Table RegionSize</title>
-      <para>The regionsize can be set on a per-table basis via <code>setFileSize</code> on <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HTableDescriptor.html">HTableDescriptor</link>
-        in the event where certain tables require different regionsizes than the configured default
-        regionsize. </para>
-      <para>See <xref
-          linkend="ops.capacity.regions" /> for more information. </para>
-    </section>
-    <section
-      xml:id="schema.bloom">
-      <title>Bloom Filters</title>
-      <para>A Bloom filter, named for its creator, Burton Howard Bloom, is a data structure which is
-        designed to predict whether a given element is a member of a set of data. A positive result
-        from a Bloom filter is not always accurate, but a negative result is guaranteed to be
-        accurate. Bloom filters are designed to be "accurate enough" for sets of data which are so
-        large that conventional hashing mechanisms would be impractical. For more information about
-        Bloom filters in general, refer to <link
-          xlink:href="http://en.wikipedia.org/wiki/Bloom_filter" />.</para>
-      <para>In terms of HBase, Bloom filters provide a lightweight in-memory structure to reduce the
-        number of disk reads for a given Get operation (Bloom filters do not work with Scans) to only the StoreFiles likely to
-        contain the desired Row. The potential performance gain increases with the number of
-        parallel reads. </para>
-      <para>The Bloom filters themselves are stored in the metadata of each HFile and never need to
-        be updated. When an HFile is opened because a region is deployed to a RegionServer, the
-        Bloom filter is loaded into memory. </para>
-      <para>HBase includes some tuning mechanisms for folding the Bloom filter to reduce the size
-        and keep the false positive rate within a desired range.</para>
-      <para>Bloom filters were introduced in <link
-          xlink:href="https://issues.apache.org/jira/browse/HBASE-1200">HBASE-1200</link>. Since
-        HBase 0.96, row-based Bloom filters are enabled by default. (<link
-          xlink:href="https://issues.apache.org/jira/browse/HBASE-8450">HBASE-</link>)</para>
-      <para>For more information on Bloom filters in relation to HBase, see <xref
-          linkend="blooms" /> for more information, or the following Quora discussion: <link
-          xlink:href="http://www.quora.com/How-are-bloom-filters-used-in-HBase">How are bloom
-          filters used in HBase?</link>. </para>
-      
-      <section xml:id="bloom.filters.when">
-        <title>When To Use Bloom Filters</title>
-        <para>Since HBase 0.96, row-based Bloom filters are enabled by default. You may choose to
-          disable them or to change some tables to use row+column Bloom filters, depending on the
-          characteristics of your data and how it is loaded into HBase.</para>
-
-        <para>To determine whether Bloom filters could have a positive impact, check the value of
-          <code>blockCacheHitRatio</code> in the RegionServer metrics. If Bloom filters are enabled, the value of
-          <code>blockCacheHitRatio</code> should increase, because the Bloom filter is filtering out blocks that
-          are definitely not needed. </para>
-        <para>You can choose to enable Bloom filters for a row or for a row+column combination. If
-          you generally scan entire rows, the row+column combination will not provide any benefit. A
-          row-based Bloom filter can operate on a row+column Get, but not the other way around.
-          However, if you have a large number of column-level Puts, such that a row may be present
-          in every StoreFile, a row-based filter will always return a positive result and provide no
-          benefit. Unless you have one column per row, row+column Bloom filters require more space,
-          in order to store more keys. Bloom filters work best when the size of each data entry is
-          at least a few kilobytes in size. </para>
-        <para>Overhead will be reduced when your data is stored in a few larger StoreFiles, to avoid
-          extra disk IO during low-level scans to find a specific row. </para>
-        <para>Bloom filters need to be rebuilt upon deletion, so may not be appropriate in
-          environments with a large number of deletions.</para>
-      </section>
-      
-      <section>
-        <title>Enabling Bloom Filters</title>
-        <para>Bloom filters are enabled on a Column Family. You can do this by using the
-          setBloomFilterType method of HColumnDescriptor or using the HBase API. Valid values are
-            <literal>NONE</literal> (the default), <literal>ROW</literal>, or
-            <literal>ROWCOL</literal>. See <xref
-            linkend="bloom.filters.when" /> for more information on <literal>ROW</literal> versus
-            <literal>ROWCOL</literal>. See also the API documentation for <link
-            xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HColumnDescriptor.html">HColumnDescriptor</link>.</para>
-          <para>The following example creates a table and enables a ROWCOL Bloom filter on the
-            <literal>colfam1</literal> column family.</para>
-        <screen>
-hbase> <userinput>create 'mytable',{NAME => 'colfam1', BLOOMFILTER => 'ROWCOL'}</userinput>          
-        </screen>
-      </section>
-      
-      <section>
-        <title>Configuring Server-Wide Behavior of Bloom Filters</title>
-        <para>You can configure the following settings in the <filename>hbase-site.xml</filename>.
-        </para>
-        <informaltable>
-          <tgroup cols="3">
-            <thead>
-              <row>
-                <entry>Parameter</entry>
-                <entry>Default</entry>
-                <entry>Description</entry>
-              </row>
-            </thead>
-            <tbody>
-              <row>
-                <entry><para><code>io.hfile.bloom.enabled</code></para></entry>
-                <entry><para><literal>yes</literal></para></entry>
-                <entry><para>Set to <literal>no</literal> to kill bloom filters server-wide if
-                    something goes wrong</para></entry>
-              </row>
-              <row>
-                <entry><para><code>io.hfile.bloom.error.rate</code></para></entry>
-                <entry><para><literal>.01</literal></para></entry>
-                <entry><para>The average false positive rate for bloom filters. Folding is used to
-                  maintain the false positive rate. Expressed as a decimal representation of a
-                  percentage.</para></entry>
-              </row>
-              <row>
-                <entry><para><code>io.hfile.bloom.max.fold</code></para></entry>
-                <entry><para><literal>7</literal></para></entry>
-                <entry><para>The guaranteed maximum fold rate. Changing this setting should not be
-                  necessary and is not recommended.</para></entry>
-              </row>
-              <row>
-                <entry><para><code>io.storefile.bloom.max.keys</code></para></entry>
-                <entry><para><literal>128000000</literal></para></entry>
-                <entry><para>For default (single-block) Bloom filters, this specifies the maximum
-                    number of keys.</para></entry>
-              </row>
-              <row>
-                <entry><para><code>io.storefile.delete.family.bloom.enabled</code></para></entry>
-                <entry><para><literal>true</literal></para></entry>
-                <entry><para>Master switch to enable Delete Family Bloom filters and store them in
-                  the StoreFile.</para></entry>
-              </row>
-              <row>
-                <entry><para><code>io.storefile.bloom.block.size</code></para></entry>
-                <entry><para><literal>65536</literal></para></entry>
-                <entry><para>Target Bloom block size. Bloom filter blocks of approximately this size
-                    are interleaved with data blocks.</para></entry>
-              </row>
-              <row>
-                <entry><para><code>hfile.block.bloom.cacheonwrite</code></para></entry>
-                <entry><para><literal>false</literal></para></entry>
-                <entry><para>Enables cache-on-write for inline blocks of a compound Bloom filter.</para></entry>
-              </row>
-            </tbody>
-          </tgroup>
-        </informaltable>
-      </section>
-    </section>
-    <section
-      xml:id="schema.cf.blocksize">
-      <title>ColumnFamily BlockSize</title>
-      <para>The blocksize can be configured for each ColumnFamily in a table, and this defaults to
-        64k. Larger cell values require larger blocksizes. There is an inverse relationship between
-        blocksize and the resulting StoreFile indexes (i.e., if the blocksize is doubled then the
-        resulting indexes should be roughly halved). </para>
-      <para>See <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HColumnDescriptor.html">HColumnDescriptor</link>
-        and <xref
-          linkend="store" />for more information. </para>
-    </section>
-    <section
-      xml:id="cf.in.memory">
-      <title>In-Memory ColumnFamilies</title>
-      <para>ColumnFamilies can optionally be defined as in-memory. Data is still persisted to disk,
-        just like any other ColumnFamily. In-memory blocks have the highest priority in the <xref
-          linkend="block.cache" />, but it is not a guarantee that the entire table will be in
-        memory. </para>
-      <para>See <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HColumnDescriptor.html">HColumnDescriptor</link>
-        for more information. </para>
-    </section>
-    <section
-      xml:id="perf.compression">
-      <title>Compression</title>
-      <para>Production systems should use compression with their ColumnFamily definitions. See <xref
-          linkend="compression" /> for more information. </para>
-      <section
-        xml:id="perf.compression.however">
-        <title>However...</title>
-        <para>Compression deflates data <emphasis>on disk</emphasis>. When it's in-memory (e.g., in
-          the MemStore) or on the wire (e.g., transferring between RegionServer and Client) it's
-          inflated. So while using ColumnFamily compression is a best practice, but it's not going
-          to completely eliminate the impact of over-sized Keys, over-sized ColumnFamily names, or
-          over-sized Column names. </para>
-        <para>See <xref
-            linkend="keysize" /> on for schema design tips, and <xref
-            linkend="keyvalue" /> for more information on HBase stores data internally. </para>
-      </section>
-    </section>
-  </section>
-  <!--  perf schema -->
-
-  <section
-    xml:id="perf.general">
-    <title>HBase General Patterns</title>
-    <section
-      xml:id="perf.general.constants">
-      <title>Constants</title>
-      <para>When people get started with HBase they have a tendency to write code that looks like
-        this:</para>
-      <programlisting language="java">
-Get get = new Get(rowkey);
-Result r = htable.get(get);
-byte[] b = r.getValue(Bytes.toBytes("cf"), Bytes.toBytes("attr"));  // returns current version of value
-      </programlisting>
-      <para>But especially when inside loops (and MapReduce jobs), converting the columnFamily and
-        column-names to byte-arrays repeatedly is surprisingly expensive. It's better to use
-        constants for the byte-arrays, like this:</para>
-      <programlisting language="java">
-public static final byte[] CF = "cf".getBytes();
-public static final byte[] ATTR = "attr".getBytes();
-...
-Get get = new Get(rowkey);
-Result r = htable.get(get);
-byte[] b = r.getValue(CF, ATTR);  // returns current version of value
-      </programlisting>
-    </section>
-
-  </section>
-  <section
-    xml:id="perf.writing">
-    <title>Writing to HBase</title>
-
-    <section
-      xml:id="perf.batch.loading">
-      <title>Batch Loading</title>
-      <para>Use the bulk load tool if you can. See <xref
-          linkend="arch.bulk.load" />. Otherwise, pay attention to the below. </para>
-    </section>
-    <!-- batch loading -->
-
-    <section
-      xml:id="precreate.regions">
-      <title> Table Creation: Pre-Creating Regions </title>
-      <para> Tables in HBase are initially created with one region by default. For bulk imports,
-        this means that all clients will write to the same region until it is large enough to split
-        and become distributed across the cluster. A useful pattern to speed up the bulk import
-        process is to pre-create empty regions. Be somewhat conservative in this, because too-many
-        regions can actually degrade performance. </para>
-      <para>There are two different approaches to pre-creating splits. The first approach is to rely
-        on the default <code>HBaseAdmin</code> strategy (which is implemented in
-          <code>Bytes.split</code>)... </para>
-      <programlisting language="java">
-byte[] startKey = ...;   	// your lowest key
-byte[] endKey = ...;   		// your highest key
-int numberOfRegions = ...;	// # of regions to create
-admin.createTable(table, startKey, endKey, numberOfRegions);
-      </programlisting>
-      <para>And the other approach is to define the splits yourself... </para>
-      <programlisting language="java">
-byte[][] splits = ...;   // create your own splits
-admin.createTable(table, splits);
-</programlisting>
-      <para> See <xref linkend="rowkey.regionsplits"/> for issues related to understanding your
-        keyspace and pre-creating regions. See <xref linkend="manual_region_splitting_decisions"/>
-        for discussion on manually pre-splitting regions.</para>
-    </section>
-    <section
-      xml:id="def.log.flush">
-      <title> Table Creation: Deferred Log Flush </title>
-      <para> The default behavior for Puts using the Write Ahead Log (WAL) is that
-          <classname>WAL</classname> edits will be written immediately. If deferred log flush is
-        used, WAL edits are kept in memory until the flush period. The benefit is aggregated and
-        asynchronous <classname>WAL</classname>- writes, but the potential downside is that if the
-        RegionServer goes down the yet-to-be-flushed edits are lost. This is safer, however, than
-        not using WAL at all with Puts. </para>
-      <para> Deferred log flush can be configured on tables via <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HTableDescriptor.html">HTableDescriptor</link>.
-        The default value of <varname>hbase.regionserver.optionallogflushinterval</varname> is
-        1000ms. </para>
-    </section>
-
-    <section
-      xml:id="perf.hbase.client.autoflush">
-      <title>HBase Client: AutoFlush</title>
-
-      <para>When performing a lot of Puts, make sure that setAutoFlush is set to false on your <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html">HTable</link>
-        instance. Otherwise, the Puts will be sent one at a time to the RegionServer. Puts added via
-          <code> htable.add(Put)</code> and <code> htable.add( &lt;List&gt; Put)</code> wind up in
-        the same write buffer. If <code>autoFlush = false</code>, these messages are not sent until
-        the write-buffer is filled. To explicitly flush the messages, call
-          <methodname>flushCommits</methodname>. Calling <methodname>close</methodname> on the
-          <classname>HTable</classname> instance will invoke
-        <methodname>flushCommits</methodname>.</para>
-    </section>
-    <section
-      xml:id="perf.hbase.client.putwal">
-      <title>HBase Client: Turn off WAL on Puts</title>
-      <para>A frequent request is to disable the WAL to increase performance of Puts. This is only
-        appropriate for bulk loads, as it puts your data at risk by removing the protection of the
-        WAL in the event of a region server crash. Bulk loads can be re-run in the event of a crash,
-        with little risk of data loss.</para>
-      <warning>
-        <para>If you disable the WAL for anything other than bulk loads, your data is at
-          risk.</para></warning>
-      <para>In general, it is best to use WAL for Puts, and where loading throughput is a concern to
-        use <link linkend="perf.batch.loading">bulk loading</link> techniques instead. For normal
-        Puts, you are not likely to see a performance improvement which would outweigh the risk. To
-        disable the WAL, see <xref linkend="wal.disable"/>.</para>
-    </section>
-    <section
-      xml:id="perf.hbase.client.regiongroup">
-      <title>HBase Client: Group Puts by RegionServer</title>
-      <para>In addition to using the writeBuffer, grouping <classname>Put</classname>s by
-        RegionServer can reduce the number of client RPC calls per writeBuffer flush. There is a
-        utility <classname>HTableUtil</classname> currently on TRUNK that does this, but you can
-        either copy that or implement your own version for those still on 0.90.x or earlier. </para>
-    </section>
-    <section
-      xml:id="perf.hbase.write.mr.reducer">
-      <title>MapReduce: Skip The Reducer</title>
-      <para>When writing a lot of data to an HBase table from a MR job (e.g., with <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.html">TableOutputFormat</link>),
-        and specifically where Puts are being emitted from the Mapper, skip the Reducer step. When a
-        Reducer step is used, all of the output (Puts) from the Mapper will get spooled to disk,
-        then sorted/shuffled to other Reducers that will most likely be off-node. It's far more
-        efficient to just write directly to HBase. </para>
-      <para>For summary jobs where HBase is used as a source and a sink, then writes will be coming
-        from the Reducer step (e.g., summarize values then write out result). This is a different
-        processing problem than from the the above case. </para>
-    </section>
-
-    <section
-      xml:id="perf.one.region">
-      <title>Anti-Pattern: One Hot Region</title>
-      <para>If all your data is being written to one region at a time, then re-read the section on
-        processing <link
-          linkend="timeseries">timeseries</link> data.</para>
-      <para>Also, if you are pre-splitting regions and all your data is <emphasis>still</emphasis>
-        winding up in a single region even though your keys aren't monotonically increasing, confirm
-        that your keyspace actually works with the split strategy. There are a variety of reasons
-        that regions may appear "well split" but won't work with your data. As the HBase client
-        communicates directly with the RegionServers, this can be obtained via <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#getRegionLocation%28byte[]%29">HTable.getRegionLocation</link>. </para>
-      <para>See <xref
-          linkend="precreate.regions" />, as well as <xref
-          linkend="perf.configurations" />
-      </para>
-    </section>
-
-  </section>
-  <!--  writing -->
-
-  <section
-    xml:id="perf.reading">
-    <title>Reading from HBase</title>
-    <para>The mailing list can help if you are having performance issues. For example, here is a
-      good general thread on what to look at addressing read-time issues: <link
-        xlink:href="http://search-hadoop.com/m/qOo2yyHtCC1">HBase Random Read latency >
-      100ms</link></para>
-    <section
-      xml:id="perf.hbase.client.caching">
-      <title>Scan Caching</title>
-
-      <para>If HBase is used as an input source for a MapReduce job, for example, make sure that the
-        input <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html">Scan</link>
-        instance to the MapReduce job has <methodname>setCaching</methodname> set to something
-        greater than the default (which is 1). Using the default value means that the map-task will
-        make call back to the region-server for every record processed. Setting this value to 500,
-        for example, will transfer 500 rows at a time to the client to be processed. There is a
-        cost/benefit to have the cache value be large because it costs more in memory for both
-        client and RegionServer, so bigger isn't always better.</para>
-      <section
-        xml:id="perf.hbase.client.caching.mr">
-        <title>Scan Caching in MapReduce Jobs</title>
-        <para>Scan settings in MapReduce jobs deserve special attention. Timeouts can result (e.g.,
-          UnknownScannerException) in Map tasks if it takes longer to process a batch of records
-          before the client goes back to the RegionServer for the next set of data. This problem can
-          occur because there is non-trivial processing occuring per row. If you process rows
-          quickly, set caching higher. If you process rows more slowly (e.g., lots of
-          transformations per row, writes), then set caching lower. </para>
-        <para>Timeouts can also happen in a non-MapReduce use case (i.e., single threaded HBase
-          client doing a Scan), but the processing that is often performed in MapReduce jobs tends
-          to exacerbate this issue. </para>
-      </section>
-    </section>
-    <section
-      xml:id="perf.hbase.client.selection">
-      <title>Scan Attribute Selection</title>
-
-      <para>Whenever a Scan is used to process large numbers of rows (and especially when used as a
-        MapReduce source), be aware of which attributes are selected. If <code>scan.addFamily</code>
-        is called then <emphasis>all</emphasis> of the attributes in the specified ColumnFamily will
-        be returned to the client. If only a small number of the available attributes are to be
-        processed, then only those attributes should be specified in the input scan because
-        attribute over-selection is a non-trivial performance penalty over large datasets. </para>
-    </section>
-    <section
-      xml:id="perf.hbase.client.seek">
-      <title>Avoid scan seeks</title>
-      <para>When columns are selected explicitly with <code>scan.addColumn</code>, HBase will
-        schedule seek operations to seek between the selected columns. When rows have few columns
-        and each column has only a few versions this can be inefficient. A seek operation is
-        generally slower if does not seek at least past 5-10 columns/versions or 512-1024
-        bytes.</para>
-      <para>In order to opportunistically look ahead a few columns/versions to see if the next
-        column/version can be found that way before a seek operation is scheduled, a new attribute
-          <code>Scan.HINT_LOOKAHEAD</code> can be set the on Scan object. The following code
-        instructs the RegionServer to attempt two iterations of next before a seek is
-        scheduled:</para>
-      <programlisting language="java">
-Scan scan = new Scan();
-scan.addColumn(...);
-scan.setAttribute(Scan.HINT_LOOKAHEAD, Bytes.toBytes(2));
-table.getScanner(scan);
-      </programlisting>
-    </section>
-    <section
-      xml:id="perf.hbase.mr.input">
-      <title>MapReduce - Input Splits</title>
-      <para>For MapReduce jobs that use HBase tables as a source, if there a pattern where the
-        "slow" map tasks seem to have the same Input Split (i.e., the RegionServer serving the
-        data), see the Troubleshooting Case Study in <xref
-          linkend="casestudies.slownode" />. </para>
-    </section>
-
-    <section
-      xml:id="perf.hbase.client.scannerclose">
-      <title>Close ResultScanners</title>
-
-      <para>This isn't so much about improving performance but rather <emphasis>avoiding</emphasis>
-        performance problems. If you forget to close <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/ResultScanner.html">ResultScanners</link>
-        you can cause problems on the RegionServers. Always have ResultScanner processing enclosed
-        in try/catch blocks...</para>
-      <programlisting language="java">
-Scan scan = new Scan();
-// set attrs...
-ResultScanner rs = htable.getScanner(scan);
-try {
-  for (Result r = rs.next(); r != null; r = rs.next()) {
-  // process result...
-} finally {
-  rs.close();  // always close the ResultScanner!
-}
-htable.close();
-      </programlisting>
-    </section>
-
-    <section
-      xml:id="perf.hbase.client.blockcache">
-      <title>Block Cache</title>
-
-      <para><link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html">Scan</link>
-        instances can be set to use the block cache in the RegionServer via the
-          <methodname>setCacheBlocks</methodname> method. For input Scans to MapReduce jobs, this
-        should be <varname>false</varname>. For frequently accessed rows, it is advisable to use the
-        block cache.</para>
-
-    <para>Cache more data by moving your Block Cache offheap.  See <xref linkend="offheap.blockcache" /></para>
-    </section>
-    <section
-      xml:id="perf.hbase.client.rowkeyonly">
-      <title>Optimal Loading of Row Keys</title>
-      <para>When performing a table <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html">scan</link>
-        where only the row keys are needed (no families, qualifiers, values or timestamps), add a
-        FilterList with a <varname>MUST_PASS_ALL</varname> operator to the scanner using
-          <methodname>setFilter</methodname>. The filter list should include both a <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/FirstKeyOnlyFilter.html">FirstKeyOnlyFilter</link>
-        and a <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/KeyOnlyFilter.html">KeyOnlyFilter</link>.
-        Using this filter combination will result in a worst case scenario of a RegionServer reading
-        a single value from disk and minimal network traffic to the client for a single row. </para>
-    </section>
-    <section
-      xml:id="perf.hbase.read.dist">
-      <title>Concurrency: Monitor Data Spread</title>
-      <para>When performing a high number of concurrent reads, monitor the data spread of the target
-        tables. If the target table(s) have too few regions then the reads could likely be served
-        from too few nodes. </para>
-      <para>See <xref
-          linkend="precreate.regions" />, as well as <xref
-          linkend="perf.configurations" />
-      </para>
-    </section>
-    <section
-      xml:id="blooms">
-      <title>Bloom Filters</title>
-      <para>Enabling Bloom Filters can save your having to go to disk and can help improve read
-        latencies.</para>
-      <para><link
-          xlink:href="http://en.wikipedia.org/wiki/Bloom_filter">Bloom filters</link> were developed
-        over in <link xlink:href="https://issues.apache.org/jira/browse/HBASE-1200">HBase-1200 Add
-          bloomfilters</link>. For description of the development process -- why static blooms rather than dynamic
-            -- and for an overview of the unique properties that pertain to blooms in HBase, as well
-            as possible future directions, see the <emphasis>Development Process</emphasis> section
-            of the document <link
-              xlink:href="https://issues.apache.org/jira/secure/attachment/12444007/Bloom_Filters_in_HBase.pdf">BloomFilters
-              in HBase</link> attached to <link
-              xlink:href="https://issues.apache.org/jira/browse/HBASE-1200">HBase-1200</link>.  The bloom filters described here are actually version two of blooms in HBase. In
-            versions up to 0.19.x, HBase had a dynamic bloom option based on work done by the <link
-              xlink:href="http://www.one-lab.org">European Commission One-Lab Project 034819</link>.
-            The core of the HBase bloom work was later pulled up into Hadoop to implement
-            org.apache.hadoop.io.BloomMapFile. Version 1 of HBase blooms never worked that well.
-            Version 2 is a rewrite from scratch though again it starts with the one-lab work.</para>
-        
-      <para>See also <xref
-          linkend="schema.bloom" />. </para>
-
-      <section
-        xml:id="bloom_footprint">
-        <title>Bloom StoreFile footprint</title>
-
-        <para>Bloom filters add an entry to the <classname>StoreFile</classname> general
-            <classname>FileInfo</classname> data structure and then two extra entries to the
-            <classname>StoreFile</classname> metadata section.</para>
-
-        <section>
-          <title>BloomFilter in the <classname>StoreFile</classname>
-            <classname>FileInfo</classname> data structure</title>
-
-          <para><classname>FileInfo</classname> has a <varname>BLOOM_FILTER_TYPE</varname> entry
-            which is set to <varname>NONE</varname>, <varname>ROW</varname> or
-              <varname>ROWCOL.</varname></para>
-        </section>
-
-        <section>
-          <title>BloomFilter entries in <classname>StoreFile</classname> metadata</title>
-
-          <para><varname>BLOOM_FILTER_META</varname> holds Bloom Size, Hash Function used, etc. Its
-            small in size and is cached on <classname>StoreFile.Reader</classname> load</para>
-          <para><varname>BLOOM_FILTER_DATA</varname> is the actual bloomfilter data. Obtained
-            on-demand. Stored in the LRU cache, if it is enabled (Its enabled by default).</para>
-        </section>
-      </section>
-      <section
-        xml:id="config.bloom">
-        <title>Bloom Filter Configuration</title>
-        <section>
-          <title><varname>io.hfile.bloom.enabled</varname> global kill switch</title>
-
-          <para><code>io.hfile.bloom.enabled</code> in <classname>Configuration</classname> serves
-            as the kill switch in case something goes wrong. Default =
-            <varname>true</varname>.</para>
-        </section>
-
-        <section>
-          <title><varname>io.hfile.bloom.error.rate</varname></title>
-
-          <para><varname>io.hfile.bloom.error.rate</varname> = average false positive rate. Default
-            = 1%. Decrease rate by ½ (e.g. to .5%) == +1 bit per bloom entry.</para>
-        </section>
-
-        <section>
-          <title><varname>io.hfile.bloom.max.fold</varname></title>
-
-          <para><varname>io.hfile.bloom.max.fold</varname> = guaranteed minimum fold rate. Most
-            people should leave this alone. Default = 7, or can collapse to at least 1/128th of
-            original size. See the <emphasis>Development Process</emphasis> section of the document <link
-              xlink:href="https://issues.apache.org/jira/secure/attachment/12444007/Bloom_Filters_in_HBase.pdf">BloomFilters
-              in HBase</link> for more on what this option means.</para>
-        </section>
-      </section>
-    </section>
-    <!--  bloom  -->
-    <section>
-      <title>Hedged Reads</title>
-      <para>Hedged reads are a feature of HDFS, introduced in <link
-          xlink:href="https://issues.apache.org/jira/browse/HDFS-5776">HDFS-5776</link>. Normally, a
-        single thread is spawned for each read request. However, if hedged reads are enabled, the
-        client waits some configurable amount of time, and if the read does not return, the client
-        spawns a second read request, against a different block replica of the same data. Whichever
-        read returns first is used, and the other read request is discarded. Hedged reads can be
-        helpful for times where a rare slow read is caused by a transient error such as a failing
-        disk or flaky network connection.</para>
-      <para> Because a HBase RegionServer is a HDFS client, you can enable hedged reads in HBase, by
-        adding the following properties to the RegionServer's hbase-site.xml and tuning the values
-        to suit your environment.</para>
-      <itemizedlist>
-        <title>Configuration for Hedged Reads</title>
-        <listitem>
-          <para><code>dfs.client.hedged.read.threadpool.size</code> - the number of threads
-            dedicated to servicing hedged reads. If this is set to 0 (the default), hedged reads are
-            disabled.</para>
-        </listitem>
-        <listitem>
-          <para><code>dfs.client.hedged.read.threshold.millis</code> - the number of milliseconds to
-            wait before spawning a second read thread.</para>
-        </listitem>
-      </itemizedlist>
-      <example>
-        <title>Hedged Reads Configuration Example</title>
-        <screen><![CDATA[<property>
-  <name>dfs.client.hedged.read.threadpool.size</name>
-  <value>20</value>  <!-- 20 threads -->
-</property>
-<property>
-  <name>dfs.client.hedged.read.threshold.millis</name>
-  <value>10</value>  <!-- 10 milliseconds -->
-</property>]]></screen>
-      </example>
-      <para>Use the following metrics to tune the settings for hedged reads on
-        your cluster. See <xref linkend="hbase_metrics"/>  for more information.</para>
-      <itemizedlist>
-        <title>Metrics for Hedged Reads</title>
-        <listitem>
-          <para>hedgedReadOps - the number of times hedged read threads have been triggered. This
-            could indicate that read requests are often slow, or that hedged reads are triggered too
-            quickly.</para>
-        </listitem>
-        <listitem>
-          <para>hedgeReadOpsWin - the number of times the hedged read thread was faster than the
-            original thread. This could indicate that a given RegionServer is having trouble
-            servicing requests.</para>
-        </listitem>
-      </itemizedlist>
-    </section>
-
-  </section>
-  <!--  reading -->
-
-  <section
-    xml:id="perf.deleting">
-    <title>Deleting from HBase</title>
-    <section
-      xml:id="perf.deleting.queue">
-      <title>Using HBase Tables as Queues</title>
-      <para>HBase tables are sometimes used as queues. In this case, special care must be taken to
-        regularly perform major compactions on tables used in this manner. As is documented in <xref
-          linkend="datamodel" />, marking rows as deleted creates additional StoreFiles which then
-        need to be processed on reads. Tombstones only get cleaned up with major compactions. </para>
-      <para>See also <xref
-          linkend="compaction" /> and <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#majorCompact%28java.lang.String%29">HBaseAdmin.majorCompact</link>.
-      </para>
-    </section>
-    <section
-      xml:id="perf.deleting.rpc">
-      <title>Delete RPC Behavior</title>
-      <para>Be aware that <code>htable.delete(Delete)</code> doesn't use the writeBuffer. It will
-        execute an RegionServer RPC with each invocation. For a large number of deletes, consider
-          <code>htable.delete(List)</code>. </para>
-      <para>See <link
-          xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#delete%28org.apache.hadoop.hbase.client.Delete%29" />
-      </para>
-    </section>
-  </section>
-  <!--  deleting -->
-
-  <section
-    xml:id="perf.hdfs">
-    <title>HDFS</title>
-    <para>Because HBase runs on <xref
-        linkend="arch.hdfs" /> it is important to understand how it works and how it affects HBase. </para>
-    <section
-      xml:id="perf.hdfs.curr">
-      <title>Current Issues With Low-Latency Reads</title>
-      <para>The original use-case for HDFS was batch processing. As such, there low-latency reads
-        were historically not a priority. With the increased adoption of Apache HBase this is
-        changing, and several improvements are already in development. See the <link
-          xlink:href="https://issues.apache.org/jira/browse/HDFS-1599">Umbrella Jira Ticket for HDFS
-          Improvements for HBase</link>. </para>
-    </section>
-    <section
-      xml:id="perf.hdfs.configs.localread">
-      <title>Leveraging local data</title>
-      <para>Since Hadoop 1.0.0 (also 0.22.1, 0.23.1, CDH3u3 and HDP 1.0) via <link
-          xlink:href="https://issues.apache.org/jira/browse/HDFS-2246">HDFS-2246</link>, it is
-        possible for the DFSClient to take a "short circuit" and read directly from the disk instead
-        of going through the DataNode when the data is local. What this means for HBase is that the
-        RegionServers can read directly off their machine's disks instead of having to open a socket
-        to talk to the DataNode, the former being generally much faster. See JD's <link
-              xlink:href="http://files.meetup.com/1350427/hug_ebay_jdcryans.pdf">Performance
-              Talk</link>. Also see <link
-          xlink:href="http://search-hadoop.com/m/zV6dKrLCVh1">HBase, mail # dev - read short
-          circuit</link> thread for more discussion around short circuit reads. </para>
-      <para>To enable "short circuit" reads, it will depend on your version of Hadoop. The original
-        shortcircuit read patch was much improved upon in Hadoop 2 in <link
-          xlink:href="https://issues.apache.org/jira/browse/HDFS-347">HDFS-347</link>. See <link
-          xlink:href="http://blog.cloudera.com/blog/2013/08/how-improved-short-circuit-local-reads-bring-better-performance-and-security-to-hadoop/" />
-        for details on the difference between the old and new implementations. See <link
-          xlink:href="http://archive.cloudera.com/cdh4/cdh/4/hadoop/hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Hadoop
-          shortcircuit reads configuration page</link> for how to enable the latter, better version
-        of shortcircuit. For example, here is a minimal config. enabling short-circuit reads added
-        to <filename>hbase-site.xml</filename>: </para>
-      <programlisting language="xml"><![CDATA[<property>
-  <name>dfs.client.read.shortcircuit</name>
-  <value>true</value>
-  <description>
-    This configuration parameter turns on short-circuit local reads.
-  </description>
-</property>
-<property>
-  <name>dfs.domain.socket.path</name>
-  <value>/home/stack/sockets/short_circuit_read_socket_PORT</value>
-  <description>
-    Optional.  This is a path to a UNIX domain socket that will be used for
-    communication between the DataNode and local HDFS clients.
-    If the string "_PORT" is present in this path, it will be replaced by the
-    TCP port of the DataNode.
-  </description>
-</property>]]></programlisting>
-      <para>Be careful about permissions for the directory that hosts the shared domain socket;
-        dfsclient will complain if open to other than the hbase user. </para>
-      <para>If you are running on an old Hadoop, one that is without <link
-          xlink:href="https://issues.apache.org/jira/browse/HDFS-347">HDFS-347</link> but that has
-          <link xlink:href="https://issues.apache.org/jira/browse/HDFS-2246">HDFS-2246</link>, you
-        must set two configurations. First, the hdfs-site.xml needs to be amended. Set the property
-          <varname>dfs.block.local-path-access.user</varname> to be the <emphasis>only</emphasis>
-        user that can use the shortcut. This has to be the user that started HBase. Then in
-        hbase-site.xml, set <varname>dfs.client.read.shortcircuit</varname> to be
-          <varname>true</varname>
-      </para>
-        
-      <para> Services -- at least the HBase RegionServers -- will need to be restarted in order to
-        pick up the new configurations. </para>
-      <note
-        xml:id="dfs.client.read.shortcircuit.buffer.size">
-        <title>dfs.client.read.shortcircuit.buffer.size</title>
-        <para>The default for this value is too high when running on a highly trafficed HBase. In
-          HBase, if this value has not been set, we set it down from the default of 1M to 128k
-          (Since HBase 0.98.0 and 0.96.1). See <link
-            xlink:href="https://issues.apache.org/jira/browse/HBASE-8143">HBASE-8143 HBase on Hadoop
-            2 with local short circuit reads (ssr) causes OOM</link>). The Hadoop DFSClient in HBase
-          will allocate a direct byte buffer of this size for <emphasis>each</emphasis> block it has
-          open; given HBase keeps its HDFS files open all the time, this can add up quickly.</para>
-      </note>
-    </section>
-
-    <section
-      xml:id="perf.hdfs.comp">
-      <title>Performance Comparisons of HBase vs. HDFS</title>
-      <para>A fairly common question on the dist-list is why HBase isn't as performant as HDFS files
-        in a batch context (e.g., as a MapReduce source or sink). The short answer is that HBase is
-        doing a lot more than HDFS (e.g., reading the KeyValues, returning the most current row or
-        specified timestamps, etc.), and as such HBase is 4-5 times slower than HDFS in this
-        processing context. There is room for improvement and this gap will, over time, be reduced,
-        but HDFS will always be faster in this use-case. </para>
-    </section>
-  </section>
-
-  <section
-    xml:id="perf.ec2">
-    <title>Amazon EC2</title>
-    <para>Performance questions are common on Amazon EC2 environments because it is a shared
-      environment. You will not see the same throughput as a dedicated server. In terms of running
-      tests on EC2, run them several times for the same reason (i.e., it's a shared environment and
-      you don't know what else is happening on the server). </para>
-    <para>If you are running on EC2 and post performance questions on the dist-list, please state
-      this fact up-front that because EC2 issues are practically a separate class of performance
-      issues. </para>
-  </section>
-
-  <section
-    xml:id="perf.hbase.mr.cluster">
-    <title>Collocating HBase and MapReduce</title>
-    <para>It is often recommended to have different clusters for HBase and MapReduce. A better
-      qualification of this is: don't collocate a HBase that serves live requests with a heavy MR
-      workload. OLTP and OLAP-optimized systems have conflicting requirements and one will lose to
-      the other, usually the former. For example, short latency-sensitive disk reads will have to
-      wait in line behind longer reads that are trying to squeeze out as much throughput as
-      possible. MR jobs that write to HBase will also generate flushes and compactions, which will
-      in turn invalidate blocks in the <xref
-        linkend="block.cache" />. </para>
-    <para>If you need to process the data from your live HBase cluster in MR, you can ship the
-      deltas with <xref
-        linkend="copy.table" /> or use replication to get the new data in real time on the OLAP
-      cluster. In the worst case, if you really need to collocate both, set MR to use less Map and
-      Reduce slots than you'd normally configure, possibly just one. </para>
-    <para>When HBase is used for OLAP operations, it's preferable to set it up in a hardened way
-      like configuring the ZooKeeper session timeout higher and giving more memory to the MemStores
-      (the argument being that the Block Cache won't be used much since the workloads are usually
-      long scans). </para>
-  </section>
-
-  <section
-    xml:id="perf.casestudy">
-    <title>Case Studies</title>
-    <para>For Performance and Troubleshooting Case Studies, see <xref
-        linkend="casestudies" />. </para>
-  </section>
-</chapter>

http://git-wip-us.apache.org/repos/asf/hbase/blob/09a422e8/src/main/docbkx/preface.xml
----------------------------------------------------------------------
diff --git a/src/main/docbkx/preface.xml b/src/main/docbkx/preface.xml
deleted file mode 100644
index a8f6895..0000000
--- a/src/main/docbkx/preface.xml
+++ /dev/null
@@ -1,83 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<preface
-    version="5.0"
-    xml:id="preface"
-    xmlns="http://docbook.org/ns/docbook"
-    xmlns:xlink="http://www.w3.org/1999/xlink"
-    xmlns:xi="http://www.w3.org/2001/XInclude"
-    xmlns:svg="http://www.w3.org/2000/svg"
-    xmlns:m="http://www.w3.org/1998/Math/MathML"
-    xmlns:html="http://www.w3.org/1999/xhtml"
-    xmlns:db="http://docbook.org/ns/docbook">
-    <!--
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *     http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
--->
-    <title>Preface</title>
-
-    <para>This is the official reference guide for the <link
-            xlink:href="http://hbase.apache.org/">HBase</link> version it ships with. Herein you
-        will find either the definitive documentation on an HBase topic as of its standing when the
-        referenced HBase version shipped, or it will point to the location in <link
-            xlink:href="http://hbase.apache.org/apidocs/index.html">javadoc</link>, <link
-            xlink:href="https://issues.apache.org/jira/browse/HBASE">JIRA</link> or <link
-            xlink:href="http://wiki.apache.org/hadoop/Hbase">wiki</link> where the pertinent
-        information can be found.</para>
-
-    <formalpara>
-        <title>About This Guide</title>
-        <para>This reference guide is a work in progress. The source for this guide can be found in
-            the <filename>src/main/docbkx</filename> directory of the HBase source. This reference
-            guide is marked up using <link xlink:href="http://www.docbook.org/">DocBook</link> from
-            which the the finished guide is generated as part of the 'site' build target. Run
-            <programlisting language="bourne">mvn site</programlisting> to generate this documentation. Amendments and
-            improvements to the documentation are welcomed. Click <link
-                xlink:href="https://issues.apache.org/jira/secure/CreateIssueDetails!init.jspa?pid=12310753&amp;issuetype=1&amp;components=12312132&amp;summary=SHORT+DESCRIPTION"
-                >this link</link> to file a new documentation bug against Apache HBase with some
-            values pre-selected.</para>
-    </formalpara>
-    <formalpara>
-        <title>Contributing to the Documentation</title>
-        <para>For an overview of Docbook and suggestions to get started contributing to the documentation, see <xref linkend="appendix_contributing_to_documentation" />.</para>
-    </formalpara>
-    <formalpara>
-        <title>Providing Feedback</title>
-        <para>This guide allows you to leave comments or questions on any page, using Disqus. Look
-            for the Comments area at the bottom of the page. Answering these questions is a
-            volunteer effort, and may be delayed.</para>
-    </formalpara>
-    
-    <note
-        xml:id="headsup">
-        <title>Heads-up if this is your first foray into the world of distributed
-            computing...</title>
-        <para> If this is your first foray into the wonderful world of Distributed Computing, then
-            you are in for some interesting times. First off, distributed systems are hard; making a
-            distributed system hum requires a disparate skillset that spans systems (hardware and
-            software) and networking. Your cluster' operation can hiccup because of any of a myriad
-            set of reasons from bugs in HBase itself through misconfigurations -- misconfiguration
-            of HBase but also operating system misconfigurations -- through to hardware problems
-            whether it be a bug in your network card drivers or an underprovisioned RAM bus (to
-            mention two recent examples of hardware issues that manifested as "HBase is slow"). You
-            will also need to do a recalibration if up to this your computing has been bound to a
-            single box. Here is one good starting point: <link
-                xlink:href="http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing">Fallacies
-                of Distributed Computing</link>. That said, you are welcome. Its a fun place to be.
-            Yours, the HBase Community. </para>
-    </note>
-</preface>


Mime
View raw message