hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From st...@apache.org
Subject svn commit: r1465200 [6/21] - in /hbase/hbase.apache.org/trunk: ./ book/ case_studies/ community/ configuration/ css/ developer/ getting_started/ images/ ops_mgt/ performance/ rpc/ schema_design/ security/ shell/ troubleshooting/ upgrading/
Date Sat, 06 Apr 2013 06:08:58 GMT
Modified: hbase/hbase.apache.org/trunk/book/hbase.accesscontrol.configuration.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/hbase.accesscontrol.configuration.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/hbase.accesscontrol.configuration.html (original)
+++ hbase/hbase.apache.org/trunk/book/hbase.accesscontrol.configuration.html Sat Apr  6 06:08:56 2013
@@ -8,7 +8,7 @@
         This describes how to set up Secure HBase for access control, with an
         example of granting and revoking user permission on table resources
         provided.
-    </p><div class="section" title="8.2.1.&nbsp;Prerequisites"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e4600"></a>8.2.1.&nbsp;Prerequisites</h3></div></div></div><p>
+    </p><div class="section" title="8.2.1.&nbsp;Prerequisites"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e4833"></a>8.2.1.&nbsp;Prerequisites</h3></div></div></div><p>
        You must configure HBase for secure operation. Refer to the section
        "Secure Client Access to HBase" and complete all of the steps described
        there.
@@ -19,7 +19,7 @@
        possible to subvert HBase access control via direct client access to
        ZooKeeper. Refer to the section on secure ZooKeeper configuration and
        complete all of the steps described there.
-    </p></div><div class="section" title="8.2.2.&nbsp;Overview"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e4607"></a>8.2.2.&nbsp;Overview</h3></div></div></div><p>
+    </p></div><div class="section" title="8.2.2.&nbsp;Overview"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e4840"></a>8.2.2.&nbsp;Overview</h3></div></div></div><p>
         With Secure RPC and Access Control enabled, client access to HBase is
         authenticated and user data is private unless access has been
         explicitly granted. Access to data can be granted at a table or per
@@ -29,7 +29,7 @@
         implementation for simplicity:
     </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>Row-level or per value (cell): This would require broader changes for storing the ACLs inline with rows. It is a future goal.</p></li><li class="listitem"><p>Push down of file ownership to HDFS: HBase is not designed for the case where files may have different permissions than the HBase system principal. Pushing file ownership down into HDFS would necessitate changes to core code. Also, while HDFS file ownership would make applying quotas easy, and possibly make bulk imports more straightforward, it is not clear that it would offer a more secure setup.</p></li><li class="listitem"><p>HBase managed "roles" as collections of permissions: We will not model "roles" internally in HBase to begin with. We instead allow group names to be granted permissions, which allows external modeling of roles via group membership. Groups are created and manipulated externally to HBase, via the Hadoop group
  mapping service.</p></li></ol></div><p>
 Access control mechanisms are mature and fairly standardized in the relational database world. The HBase implementation approximates current convention, but HBase has a simpler feature set than relational databases, especially in terms of client operations. We don't distinguish between an insert (new record) and update (of existing record), for example, as both collapse down into a Put. Accordingly, the important operations condense to four permissions: READ, WRITE, CREATE, and ADMIN.
-    </p><table id="d2475e4626">Operation To Permission Mapping<thead>PermissionOperation</thead><tbody>ReadGetExistsScanWritePutDeleteLock/UnlockRowIncrementColumnValueCheckAndDelete/PutFlushCompactCreateCreateAlterDropAdminEnable/DisableSplitMajor CompactGrantRevokeShutdown</tbody></table><p>
+    </p><table id="d2519e4859">Operation To Permission Mapping<thead>PermissionOperation</thead><tbody>ReadGetExistsScanWritePutDeleteLock/UnlockRowIncrementColumnValueCheckAndDelete/PutFlushCompactCreateCreateAlterDropAdminEnable/DisableSplitMajor CompactGrantRevokeShutdown</tbody></table><p>
         Permissions can be granted in any of the following scopes, though
         CREATE and ADMIN permissions are effective only at table scope.
     </p><p>
@@ -57,7 +57,7 @@ Access control mechanisms are mature and
        alter operation by setting or changing the OWNER table attribute. Only a
        single user principal can own a table at a given time. A table owner will
        have all permissions over a given table. 
-    </p></div><div class="section" title="8.2.3.&nbsp;Server-side Configuration for Access Control"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e4764"></a>8.2.3.&nbsp;Server-side Configuration for Access Control</h3></div></div></div><p>
+    </p></div><div class="section" title="8.2.3.&nbsp;Server-side Configuration for Access Control"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e4997"></a>8.2.3.&nbsp;Server-side Configuration for Access Control</h3></div></div></div><p>
         Enable the AccessController coprocessor in the cluster configuration
         and restart HBase. The restart can be a rolling one. Complete the
         restart of all Master and RegionServer processes before setting up
@@ -74,7 +74,7 @@ Access control mechanisms are mature and
         &lt;value&gt;org.apache.hadoop.hbase.security.token.TokenProvider,
         org.apache.hadoop.hbase.security.access.AccessController&lt;/value&gt;
       &lt;/property&gt;
-    </pre></div><div class="section" title="8.2.4.&nbsp;Shell Enhancements for Access Control"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e4776"></a>8.2.4.&nbsp;Shell Enhancements for Access Control</h3></div></div></div><p>
+    </pre></div><div class="section" title="8.2.4.&nbsp;Shell Enhancements for Access Control"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e5009"></a>8.2.4.&nbsp;Shell Enhancements for Access Control</h3></div></div></div><p>
 The HBase shell has been extended to provide simple commands for editing and updating user permissions. The following commands have been added for access control list management: 
     </p>
     Grant

Modified: hbase/hbase.apache.org/trunk/book/hbase.rpc.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/hbase.rpc.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/hbase.rpc.html (original)
+++ hbase/hbase.apache.org/trunk/book/hbase.rpc.html Sat Apr  6 06:08:56 2013
@@ -1,13 +1,13 @@
 <html><head>
       <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
-   <title>Appendix&nbsp;J.&nbsp;0.95 RPC Specification</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="prev" href="tracing.client.modifications.html" title="I.2.&nbsp;Client Modifications"><link rel="next" href="apjs02.html" title="J.2.&nbsp;TODO"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix&nbsp;J.&nbsp;0.95 RPC Specification</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="tracing.client.modifications.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="apjs02.htm
 l">Next</a></td></tr></table><hr></div><div class="appendix" title="Appendix&nbsp;J.&nbsp;0.95 RPC Specification"><div class="titlepage"><div><div><h2 class="title"><a name="hbase.rpc"></a>Appendix&nbsp;J.&nbsp;0.95 RPC Specification</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="hbase.rpc.html#d2475e12965">J.1. Goals</a></span></dt><dt><span class="section"><a href="apjs02.html">J.2. TODO</a></span></dt><dt><span class="section"><a href="apjs03.html">J.3. RPC</a></span></dt><dd><dl><dt><span class="section"><a href="apjs03.html#d2475e13008">J.3.1. Connection Setup</a></span></dt><dt><span class="section"><a href="apjs03.html#d2475e13040">J.3.2. Request</a></span></dt><dt><span class="section"><a href="apjs03.html#d2475e13073">J.3.3. Response</a></span></dt><dt><span class="section"><a href="apjs03.html#d2475e13103">J.3.4. Exceptions</a></span></dt><dt><span class="section"><a href="apjs03.html#d2475e13110">J.3.5.
  CellBlocks</a></span></dt></dl></dd><dt><span class="section"><a href="apjs04.html">J.4. Notes</a></span></dt><dd><dl><dt><span class="section"><a href="apjs04.html#d2475e13118">J.4.1. Constraints</a></span></dt><dt><span class="section"><a href="apjs04.html#d2475e13123">J.4.2. One fat pb request or header+param</a></span></dt><dt><span class="section"><a href="apjs04.html#d2475e13143">J.4.3. Compression</a></span></dt></dl></dd></dl></div><p>In 0.95, all client/server communication is done with
+   <title>Appendix&nbsp;J.&nbsp;0.95 RPC Specification</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="prev" href="tracing.client.modifications.html" title="I.2.&nbsp;Client Modifications"><link rel="next" href="apjs02.html" title="J.2.&nbsp;TODO"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix&nbsp;J.&nbsp;0.95 RPC Specification</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="tracing.client.modifications.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="apjs02.htm
 l">Next</a></td></tr></table><hr></div><div class="appendix" title="Appendix&nbsp;J.&nbsp;0.95 RPC Specification"><div class="titlepage"><div><div><h2 class="title"><a name="hbase.rpc"></a>Appendix&nbsp;J.&nbsp;0.95 RPC Specification</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="hbase.rpc.html#d2519e13171">J.1. Goals</a></span></dt><dt><span class="section"><a href="apjs02.html">J.2. TODO</a></span></dt><dt><span class="section"><a href="apjs03.html">J.3. RPC</a></span></dt><dd><dl><dt><span class="section"><a href="apjs03.html#d2519e13214">J.3.1. Connection Setup</a></span></dt><dt><span class="section"><a href="apjs03.html#d2519e13246">J.3.2. Request</a></span></dt><dt><span class="section"><a href="apjs03.html#d2519e13279">J.3.3. Response</a></span></dt><dt><span class="section"><a href="apjs03.html#d2519e13309">J.3.4. Exceptions</a></span></dt><dt><span class="section"><a href="apjs03.html#d2519e13316">J.3.5.
  CellBlocks</a></span></dt></dl></dd><dt><span class="section"><a href="apjs04.html">J.4. Notes</a></span></dt><dd><dl><dt><span class="section"><a href="apjs04.html#d2519e13324">J.4.1. Constraints</a></span></dt><dt><span class="section"><a href="apjs04.html#d2519e13329">J.4.2. One fat pb request or header+param</a></span></dt><dt><span class="section"><a href="apjs04.html#d2519e13349">J.4.3. Compression</a></span></dt></dl></dd></dl></div><p>In 0.95, all client/server communication is done with
       <a class="link" href="https://code.google.com/p/protobuf/" target="_top">protobuf&#8217;ed</a> Messages rather than with
       <a class="link" href="http://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/Writable.html" target="_top">Hadoop Writables</a>.
       Our RPC wire format therefore changes.
       This document describes the client/server request/response protocol and our new RPC wire-format.</p><p></p><p>For what RPC is like in 0.94 and previous,
       see Beno&icirc;t/Tsuna&#8217;s <a class="link" href="https://github.com/OpenTSDB/asynchbase/blob/master/src/HBaseRpc.java#L164" target="_top">Unofficial Hadoop / HBase RPC protocol documentation</a>.
       For more background on how we arrived at this spec., see
-      <a class="link" href="https://docs.google.com/document/d/1WCKwgaLDqBw2vpux0jPsAu2WPTRISob7HGCO8YhfDTA/edit#" target="_top">HBase RPC: WIP</a></p><p></p><div class="section" title="J.1.&nbsp;Goals"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d2475e12965"></a>J.1.&nbsp;Goals</h2></div></div></div><p>
+      <a class="link" href="https://docs.google.com/document/d/1WCKwgaLDqBw2vpux0jPsAu2WPTRISob7HGCO8YhfDTA/edit#" target="_top">HBase RPC: WIP</a></p><p></p><div class="section" title="J.1.&nbsp;Goals"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d2519e13171"></a>J.1.&nbsp;Goals</h2></div></div></div><p>
       </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>A wire-format we can evolve</p></li><li class="listitem"><p>A format that does not require our rewriting server core or
                   radically changing its current architecture (for later).</p></li></ol></div><p>
   </p></div></div><div id="disqus_thread"></div><script type="text/javascript">

Modified: hbase/hbase.apache.org/trunk/book/hbase.tests.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/hbase.tests.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/hbase.tests.html (original)
+++ hbase/hbase.apache.org/trunk/book/hbase.tests.html Sat Apr  6 06:08:56 2013
@@ -46,18 +46,18 @@ Apache HBase uses a patched maven surefi
 its unit test characterizations.
 </p><p>Read the below to figure which annotation of the set small, medium, and large to
 put on your new HBase unit test.
-</p><div class="section" title="15.7.2.1.&nbsp;Small Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.small"></a>15.7.2.1.&nbsp;Small Tests<a class="indexterm" name="d2475e10400"></a></h4></div></div></div><p>
+</p><div class="section" title="15.7.2.1.&nbsp;Small Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.small"></a>15.7.2.1.&nbsp;Small Tests<a class="indexterm" name="d2519e10581"></a></h4></div></div></div><p>
 <span class="emphasis"><em>Small</em></span> tests are executed in a shared JVM. We put in this category all the tests that can
 be executed quickly in a shared JVM.  The maximum execution time for a small test is 15 seconds,
-and small tests should not use a (mini)cluster.</p></div><div class="section" title="15.7.2.2.&nbsp;Medium Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.medium"></a>15.7.2.2.&nbsp;Medium Tests<a class="indexterm" name="d2475e10411"></a></h4></div></div></div><p><span class="emphasis"><em>Medium</em></span> tests represent tests that must be executed
+and small tests should not use a (mini)cluster.</p></div><div class="section" title="15.7.2.2.&nbsp;Medium Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.medium"></a>15.7.2.2.&nbsp;Medium Tests<a class="indexterm" name="d2519e10592"></a></h4></div></div></div><p><span class="emphasis"><em>Medium</em></span> tests represent tests that must be executed
 before proposing a patch. They are designed to run in less than 30 minutes altogether,
 and are quite stable in their results. They are designed to last less than 50 seconds
 individually. They can use a cluster, and each of them is executed in a separate JVM.
-</p></div><div class="section" title="15.7.2.3.&nbsp;Large Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.large"></a>15.7.2.3.&nbsp;Large Tests<a class="indexterm" name="d2475e10421"></a></h4></div></div></div><p><span class="emphasis"><em>Large</em></span> tests are everything else. They are typically large-scale
+</p></div><div class="section" title="15.7.2.3.&nbsp;Large Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.large"></a>15.7.2.3.&nbsp;Large Tests<a class="indexterm" name="d2519e10602"></a></h4></div></div></div><p><span class="emphasis"><em>Large</em></span> tests are everything else. They are typically large-scale
 tests, regression tests for specific bugs, timeout tests, performance tests.
 They are executed before a commit on the pre-integration machines. They can be run on
 the developer machine as well.
-</p></div><div class="section" title="15.7.2.4.&nbsp;Integration Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.integration"></a>15.7.2.4.&nbsp;Integration Tests<a class="indexterm" name="d2475e10431"></a></h4></div></div></div><p><span class="emphasis"><em>Integration</em></span> tests are system level tests. See
+</p></div><div class="section" title="15.7.2.4.&nbsp;Integration Tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.integration"></a>15.7.2.4.&nbsp;Integration Tests<a class="indexterm" name="d2519e10612"></a></h4></div></div></div><p><span class="emphasis"><em>Integration</em></span> tests are system level tests. See
 <a class="xref" href="hbase.tests.html#integration.tests" title="15.7.5.&nbsp;Integration Tests">Section&nbsp;15.7.5, &#8220;Integration Tests&#8221;</a> for more info.
 </p></div></div><div class="section" title="15.7.3.&nbsp;Running tests"><div class="titlepage"><div><div><h3 class="title"><a name="hbase.unittests.cmds"></a>15.7.3.&nbsp;Running tests</h3></div></div></div><p>Below we describe how to run the Apache HBase junit categories.</p><div class="section" title="15.7.3.1.&nbsp;Default: small and medium category tests"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.cmds.test"></a>15.7.3.1.&nbsp;Default: small and medium category tests
 </h4></div></div></div><p>Running </p><pre class="programlisting">mvn test</pre><p> will execute all small tests in a single JVM
@@ -112,7 +112,7 @@ It must be executed from the directory w
 Running </p><pre class="programlisting">./dev-support/hbasetests.sh runAllTests</pre><p> will execute all tests.
 Running </p><pre class="programlisting">./dev-support/hbasetests.sh replayFailed</pre><p> will rerun the failed tests a
 second time, in a separate jvm and without parallelisation.
-</p></div><div class="section" title="15.7.3.7.&nbsp;Test Resource Checker"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.resource.checker"></a>15.7.3.7.&nbsp;Test Resource Checker<a class="indexterm" name="d2475e10559"></a></h4></div></div></div><p>
+</p></div><div class="section" title="15.7.3.7.&nbsp;Test Resource Checker"><div class="titlepage"><div><div><h4 class="title"><a name="hbase.unittests.resource.checker"></a>15.7.3.7.&nbsp;Test Resource Checker<a class="indexterm" name="d2519e10740"></a></h4></div></div></div><p>
 A custom Maven SureFire plugin listener checks a  number of resources before
 and after each HBase unit test runs and logs its findings at the end of the test
 output files which can be found in <code class="filename">target/surefire-reports</code>

Modified: hbase/hbase.apache.org/trunk/book/hbck.in.depth.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/hbck.in.depth.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/hbck.in.depth.html (original)
+++ hbase/hbase.apache.org/trunk/book/hbck.in.depth.html Sat Apr  6 06:08:56 2013
@@ -1,9 +1,9 @@
 <html><head>
       <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
-   <title>Appendix&nbsp;B.&nbsp;hbck In Depth</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="prev" href="faq.html" title="Appendix&nbsp;A.&nbsp;FAQ"><link rel="next" href="apbs02.html" title="B.2.&nbsp;Inconsistencies"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix&nbsp;B.&nbsp;hbck In Depth</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="faq.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="apbs02.html">Next</a></td></tr></table><hr></div><div class="appendix" t
 itle="Appendix&nbsp;B.&nbsp;hbck In Depth"><div class="titlepage"><div><div><h2 class="title"><a name="hbck.in.depth"></a>Appendix&nbsp;B.&nbsp;hbck In Depth</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="hbck.in.depth.html#d2475e11964">B.1. Running hbck to identify inconsistencies</a></span></dt><dt><span class="section"><a href="apbs02.html">B.2. Inconsistencies</a></span></dt><dt><span class="section"><a href="apbs03.html">B.3. Localized repairs</a></span></dt><dt><span class="section"><a href="apbs04.html">B.4. Region Overlap Repairs</a></span></dt><dd><dl><dt><span class="section"><a href="apbs04.html#d2475e12076">B.4.1. Special cases: Meta is not properly assigned</a></span></dt><dt><span class="section"><a href="apbs04.html#d2475e12085">B.4.2. Special cases: HBase version file is missing</a></span></dt><dt><span class="section"><a href="apbs04.html#d2475e12092">B.4.3. Special case: Root and META are corrupt
 .</a></span></dt><dt><span class="section"><a href="apbs04.html#d2475e12099">B.4.4. Special cases: Offline split parent</a></span></dt></dl></dd></dl></div><p>HBaseFsck (hbck) is a tool for checking for region consistency and table integrity problems
+   <title>Appendix&nbsp;B.&nbsp;hbck In Depth</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="prev" href="faq.html" title="Appendix&nbsp;A.&nbsp;FAQ"><link rel="next" href="apbs02.html" title="B.2.&nbsp;Inconsistencies"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix&nbsp;B.&nbsp;hbck In Depth</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="faq.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="apbs02.html">Next</a></td></tr></table><hr></div><div class="appendix" t
 itle="Appendix&nbsp;B.&nbsp;hbck In Depth"><div class="titlepage"><div><div><h2 class="title"><a name="hbck.in.depth"></a>Appendix&nbsp;B.&nbsp;hbck In Depth</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="hbck.in.depth.html#d2519e12170">B.1. Running hbck to identify inconsistencies</a></span></dt><dt><span class="section"><a href="apbs02.html">B.2. Inconsistencies</a></span></dt><dt><span class="section"><a href="apbs03.html">B.3. Localized repairs</a></span></dt><dt><span class="section"><a href="apbs04.html">B.4. Region Overlap Repairs</a></span></dt><dd><dl><dt><span class="section"><a href="apbs04.html#d2519e12282">B.4.1. Special cases: Meta is not properly assigned</a></span></dt><dt><span class="section"><a href="apbs04.html#d2519e12291">B.4.2. Special cases: HBase version file is missing</a></span></dt><dt><span class="section"><a href="apbs04.html#d2519e12298">B.4.3. Special case: Root and META are corrupt
 .</a></span></dt><dt><span class="section"><a href="apbs04.html#d2519e12305">B.4.4. Special cases: Offline split parent</a></span></dt></dl></dd></dl></div><p>HBaseFsck (hbck) is a tool for checking for region consistency and table integrity problems
 and repairing a corrupted HBase. It works in two basic modes -- a read-only inconsistency
 identifying mode and a multi-phase read-write repair mode.
-	</p><div class="section" title="B.1.&nbsp;Running hbck to identify inconsistencies"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d2475e11964"></a>B.1.&nbsp;Running hbck to identify inconsistencies</h2></div></div></div>
+	</p><div class="section" title="B.1.&nbsp;Running hbck to identify inconsistencies"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d2519e12170"></a>B.1.&nbsp;Running hbck to identify inconsistencies</h2></div></div></div>
 To check to see if your HBase cluster has corruptions, run hbck against your HBase cluster:
 <pre class="programlisting">
 $ ./bin/hbase hbck

Modified: hbase/hbase.apache.org/trunk/book/hfilev2.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/hfilev2.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/hfilev2.html (original)
+++ hbase/hbase.apache.org/trunk/book/hfilev2.html Sat Apr  6 06:08:56 2013
@@ -1,12 +1,12 @@
 <html><head>
       <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
-   <title>Appendix&nbsp;E.&nbsp;HFile format version 2</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="prev" href="apd.html" title="Appendix&nbsp;D.&nbsp;YCSB: The Yahoo! Cloud Serving Benchmark and HBase"><link rel="next" href="apes02.html" title="E.2.&nbsp;HFile format version 1 overview"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix&nbsp;E.&nbsp;HFile format version 2</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="apd.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="
 n" href="apes02.html">Next</a></td></tr></table><hr></div><div class="appendix" title="Appendix&nbsp;E.&nbsp;HFile format version 2"><div class="titlepage"><div><div><h2 class="title"><a name="hfilev2"></a>Appendix&nbsp;E.&nbsp;HFile format version 2</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="hfilev2.html#d2475e12260">E.1. Motivation </a></span></dt><dt><span class="section"><a href="apes02.html">E.2. HFile format version 1 overview </a></span></dt><dd><dl><dt><span class="section"><a href="apes02.html#d2475e12295">E.2.1.  Block index format in version 1 </a></span></dt></dl></dd><dt><span class="section"><a href="apes03.html">E.3. 
+   <title>Appendix&nbsp;E.&nbsp;HFile format version 2</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="prev" href="apd.html" title="Appendix&nbsp;D.&nbsp;YCSB: The Yahoo! Cloud Serving Benchmark and HBase"><link rel="next" href="apes02.html" title="E.2.&nbsp;HFile format version 1 overview"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Appendix&nbsp;E.&nbsp;HFile format version 2</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="apd.html">Prev</a>&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="
 n" href="apes02.html">Next</a></td></tr></table><hr></div><div class="appendix" title="Appendix&nbsp;E.&nbsp;HFile format version 2"><div class="titlepage"><div><div><h2 class="title"><a name="hfilev2"></a>Appendix&nbsp;E.&nbsp;HFile format version 2</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="hfilev2.html#d2519e12466">E.1. Motivation </a></span></dt><dt><span class="section"><a href="apes02.html">E.2. HFile format version 1 overview </a></span></dt><dd><dl><dt><span class="section"><a href="apes02.html#d2519e12501">E.2.1.  Block index format in version 1 </a></span></dt></dl></dd><dt><span class="section"><a href="apes03.html">E.3. 
       HBase file format with inline blocks (version 2)
-      </a></span></dt><dd><dl><dt><span class="section"><a href="apes03.html#d2475e12322">E.3.1.  Overview</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12337">E.3.2. Unified version 2 block format</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12406">E.3.3.  Block index in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12431">E.3.4. 
-      Root block index format in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12484">E.3.5. 
-      Non-root block index format in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12509">E.3.6. 
-      Bloom filters in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12546">E.3.7. File Info format in versions 1 and 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2475e12592">E.3.8. 
-      Fixed file trailer format differences between versions 1 and 2</a></span></dt></dl></dd></dl></div><div class="section" title="E.1.&nbsp;Motivation"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d2475e12260"></a>E.1.&nbsp;Motivation </h2></div></div></div><p>Note:  this feature was introduced in HBase 0.92</p><p>We found it necessary to revise the HFile format after encountering high memory usage and slow startup times caused by large Bloom filters and block indexes in the region server. Bloom filters can get as large as 100 MB per HFile, which adds up to 2 GB when aggregated over 20 regions. Block indexes can grow as large as 6 GB in aggregate size over the same set of regions. A region is not considered opened until all of its block index data is loaded. Large Bloom filters produce a different performance problem: the first get request that requires a Bloom filter lookup will incur the latency of loading the entire Bloom filter bit a
 rray.</p><p>To speed up region server startup we break Bloom filters and block indexes into multiple blocks and write those blocks out as they fill up, which also reduces the HFile writer&#8217;s memory footprint. In the Bloom filter case, &#8220;filling up a block&#8221; means accumulating enough keys to efficiently utilize a fixed-size bit array, and in the block index case we accumulate an &#8220;index block&#8221; of the desired size. Bloom filter blocks and index blocks (we call these &#8220;inline blocks&#8221;) become interspersed with data blocks, and as a side effect we can no longer rely on the difference between block offsets to determine data block length, as it was done in version 1.</p><p>HFile is a low-level file format by design, and it should not deal with application-specific details such as Bloom filters, which are handled at StoreFile level. Therefore, we call Bloom filter blocks in an HFile "inline" blocks. We also supply HFile with an interface to write
  those inline blocks. </p><p>Another format modification aimed at reducing the region server startup time is to use a contiguous &#8220;load-on-open&#8221; section that has to be loaded in memory at the time an HFile is being opened. Currently, as an HFile opens, there are separate seek operations to read the trailer, data/meta indexes, and file info. To read the Bloom filter, there are two more seek operations for its &#8220;data&#8221; and &#8220;meta&#8221; portions. In version 2, we seek once to read the trailer and seek again to read everything else we need to open the file from a contiguous block.</p></div></div><div id="disqus_thread"></div><script type="text/javascript">
+      </a></span></dt><dd><dl><dt><span class="section"><a href="apes03.html#d2519e12528">E.3.1.  Overview</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12543">E.3.2. Unified version 2 block format</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12612">E.3.3.  Block index in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12637">E.3.4. 
+      Root block index format in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12690">E.3.5. 
+      Non-root block index format in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12715">E.3.6. 
+      Bloom filters in version 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12752">E.3.7. File Info format in versions 1 and 2</a></span></dt><dt><span class="section"><a href="apes03.html#d2519e12798">E.3.8. 
+      Fixed file trailer format differences between versions 1 and 2</a></span></dt></dl></dd></dl></div><div class="section" title="E.1.&nbsp;Motivation"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="d2519e12466"></a>E.1.&nbsp;Motivation </h2></div></div></div><p>Note:  this feature was introduced in HBase 0.92</p><p>We found it necessary to revise the HFile format after encountering high memory usage and slow startup times caused by large Bloom filters and block indexes in the region server. Bloom filters can get as large as 100 MB per HFile, which adds up to 2 GB when aggregated over 20 regions. Block indexes can grow as large as 6 GB in aggregate size over the same set of regions. A region is not considered opened until all of its block index data is loaded. Large Bloom filters produce a different performance problem: the first get request that requires a Bloom filter lookup will incur the latency of loading the entire Bloom filter bit a
 rray.</p><p>To speed up region server startup we break Bloom filters and block indexes into multiple blocks and write those blocks out as they fill up, which also reduces the HFile writer&#8217;s memory footprint. In the Bloom filter case, &#8220;filling up a block&#8221; means accumulating enough keys to efficiently utilize a fixed-size bit array, and in the block index case we accumulate an &#8220;index block&#8221; of the desired size. Bloom filter blocks and index blocks (we call these &#8220;inline blocks&#8221;) become interspersed with data blocks, and as a side effect we can no longer rely on the difference between block offsets to determine data block length, as it was done in version 1.</p><p>HFile is a low-level file format by design, and it should not deal with application-specific details such as Bloom filters, which are handled at StoreFile level. Therefore, we call Bloom filter blocks in an HFile "inline" blocks. We also supply HFile with an interface to write
  those inline blocks. </p><p>Another format modification aimed at reducing the region server startup time is to use a contiguous &#8220;load-on-open&#8221; section that has to be loaded in memory at the time an HFile is being opened. Currently, as an HFile opens, there are separate seek operations to read the trailer, data/meta indexes, and file info. To read the Bloom filter, there are two more seek operations for its &#8220;data&#8221; and &#8220;meta&#8221; portions. In version 2, we seek once to read the trailer and seek again to read everything else we need to open the file from a contiguous block.</p></div></div><div id="disqus_thread"></div><script type="text/javascript">
     var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
     var disqus_url = 'http://hbase.apache.org/book';
     var disqus_identifier = 'hfilev2';

Modified: hbase/hbase.apache.org/trunk/book/important_configurations.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/important_configurations.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/important_configurations.html (original)
+++ hbase/hbase.apache.org/trunk/book/important_configurations.html Sat Apr  6 06:08:56 2013
@@ -120,7 +120,7 @@
           Keeping 5 regions per RS would be too low for a job, whereas 1000 will generate too many maps.
       </p></div></div><div class="section" title="2.5.2.7.&nbsp;Managed Splitting"><div class="titlepage"><div><div><h4 class="title"><a name="disable.splitting"></a>2.5.2.7.&nbsp;Managed Splitting</h4></div></div></div><p>
       Rather than let HBase auto-split your Regions, manage the splitting manually
-      <sup>[<a name="d2475e2547" href="#ftn.d2475e2547" class="footnote">11</a>]</sup>.
+      <sup>[<a name="d2519e2547" href="#ftn.d2519e2547" class="footnote">11</a>]</sup>.
  With growing amounts of data, splits will continually be needed. Since
  you always know exactly what regions you have, long-term debugging and
  profiling is much easier with manual splits. It is hard to trace the logs to
@@ -180,7 +180,7 @@ of all regions.
       <a class="link" href="http://search-hadoop.com/m/pduLg2fydtE/Inconsistent+scan+performance+with+caching+set+&amp;subj=Re+Inconsistent+scan+performance+with+caching+set+to+1" target="_top">Inconsistent scan performance with caching set to 1</a>
       and the issue cited therein where setting notcpdelay improved scan speeds.  You might also
       see the graphs on the tail of <a class="link" href="https://issues.apache.org/jira/browse/HBASE-7008" target="_top">HBASE-7008 Set scanner caching to a better default</a>
-      where our Lars Hofhansl tries various data sizes w/ Nagle's on and off measuring the effect.</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2475e2547" href="#d2475e2547" class="para">11</a>] </sup>What follows is taken from the javadoc at the head of
+      where our Lars Hofhansl tries various data sizes w/ Nagle's on and off measuring the effect.</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2519e2547" href="#d2519e2547" class="para">11</a>] </sup>What follows is taken from the javadoc at the head of
       the <code class="classname">org.apache.hadoop.hbase.util.RegionSplitter</code> tool
       added to HBase post-0.90.0 release.
       </p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">

Modified: hbase/hbase.apache.org/trunk/book/jvm.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/jvm.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/jvm.html (original)
+++ hbase/hbase.apache.org/trunk/book/jvm.html Sat Apr  6 06:08:56 2013
@@ -10,11 +10,11 @@
         from defaults. Start at 60 or 70 percent (The lower you bring down the
         threshold, the more GCing is done, the more CPU used). To address the
         second fragmentation issue, Todd added an experimental facility,
-        <a class="indexterm" name="d2475e7111"></a>, that
+        <a class="indexterm" name="d2519e7344"></a>, that
         must be explicitly enabled in Apache HBase 0.90.x (Its defaulted to be on in
         Apache 0.92.x HBase). See <code class="code">hbase.hregion.memstore.mslab.enabled</code>
         to true in your <code class="classname">Configuration</code>. See the cited
-        slides for background and detail<sup>[<a name="d2475e7121" href="#ftn.d2475e7121" class="footnote">24</a>]</sup>.
+        slides for background and detail<sup>[<a name="d2519e7354" href="#ftn.d2519e7354" class="footnote">25</a>]</sup>.
         Be aware that when enabled, each MemStore instance will occupy at least
         an MSLAB instance of memory.  If you have thousands of regions or lots
         of regions each with many column families, this allocation of MSLAB
@@ -22,7 +22,7 @@
         an extreme case cause you to OOME.  Disable MSLAB in this case, or
         lower the amount of memory it uses or float less regions per server.
         </p><p>For more information about GC logs, see <a class="xref" href="trouble.log.html#trouble.log.gc" title="12.2.3.&nbsp;JVM Garbage Collection Logs">Section&nbsp;12.2.3, &#8220;JVM Garbage Collection Logs&#8221;</a>.
-        </p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2475e7121" href="#d2475e7121" class="para">24</a>] </sup>The latest jvms do better
+        </p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2519e7354" href="#d2519e7354" class="para">25</a>] </sup>The latest jvms do better
         regards fragmentation so make sure you are running a recent release.
         Read down in the message,
         <a class="link" href="http://osdir.com/ml/hotspot-gc-use/2011-11/msg00002.html" target="_top">Identifying concurrent mode failures caused by fragmentation</a>.</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">

Modified: hbase/hbase.apache.org/trunk/book/master.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/master.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/master.html (original)
+++ hbase/hbase.apache.org/trunk/book/master.html Sat Apr  6 06:08:56 2013
@@ -2,7 +2,7 @@
       <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <title>9.5.&nbsp;Master</title><link rel="stylesheet" type="text/css" href="../css/freebsd_docbook.css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"><link rel="home" href="book.html" title="The Apache HBase&#153; Reference Guide"><link rel="up" href="architecture.html" title="Chapter&nbsp;9.&nbsp;Architecture"><link rel="prev" href="client.filter.html" title="9.4.&nbsp;Client Request Filters"><link rel="next" href="regionserver.arch.html" title="9.6.&nbsp;RegionServer"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">9.5.&nbsp;Master</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="client.filter.html">Prev</a>&nbsp;</td><th width="60%" align="center">Chapter&nbsp;9.&nbsp;Architecture</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="regionserver.arch.html">Next</a></td></tr></t
 able><hr></div><div class="section" title="9.5.&nbsp;Master"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="master"></a>9.5.&nbsp;Master</h2></div></div></div><p><code class="code">HMaster</code> is the implementation of the Master Server.  The Master server
        is responsible for monitoring all RegionServer instances in the cluster, and is
-       the interface for all metadata changes.  In a distributed cluster, the Master typically runs on the <a class="xref" href="arch.hdfs.html#arch.hdfs.nn" title="9.9.1.&nbsp;NameNode">Section&nbsp;9.9.1, &#8220;NameNode&#8221;</a><sup>[<a name="d2475e5366" href="#ftn.d2475e5366" class="footnote">21</a>]</sup>
+       the interface for all metadata changes.  In a distributed cluster, the Master typically runs on the <a class="xref" href="arch.hdfs.html#arch.hdfs.nn" title="9.9.1.&nbsp;NameNode">Section&nbsp;9.9.1, &#8220;NameNode&#8221;</a><sup>[<a name="d2519e5599" href="#ftn.d2519e5599" class="footnote">22</a>]</sup>
        </p><div class="section" title="9.5.1.&nbsp;Startup Behavior"><div class="titlepage"><div><div><h3 class="title"><a name="master.startup"></a>9.5.1.&nbsp;Startup Behavior</h3></div></div></div><p>If run in a multi-Master environment, all Masters compete to run the cluster.  If the active
          Master loses its lease in ZooKeeper (or the Master shuts down), then then the remaining Masters jostle to
          take over the Master role.
@@ -22,7 +22,7 @@
          </p><div class="section" title="9.5.4.1.&nbsp;LoadBalancer"><div class="titlepage"><div><div><h4 class="title"><a name="master.processes.loadbalancer"></a>9.5.4.1.&nbsp;LoadBalancer</h4></div></div></div><p>Periodically, and when there are no regions in transition,
              a load balancer will run and move regions around to balance the cluster's load.
              See <a class="xref" href="important_configurations.html#balancer_config" title="2.5.3.1.&nbsp;Balancer">Section&nbsp;2.5.3.1, &#8220;Balancer&#8221;</a> for configuring this property.</p><p>See <a class="xref" href="regions.arch.html#regions.arch.assignment" title="9.7.2.&nbsp;Region-RegionServer Assignment">Section&nbsp;9.7.2, &#8220;Region-RegionServer Assignment&#8221;</a> for more information on region assignment.
-             </p></div><div class="section" title="9.5.4.2.&nbsp;CatalogJanitor"><div class="titlepage"><div><div><h4 class="title"><a name="master.processes.catalog"></a>9.5.4.2.&nbsp;CatalogJanitor</h4></div></div></div><p>Periodically checks and cleans up the .META. table.  See <a class="xref" href="arch.catalog.html#arch.catalog.meta" title="9.2.2.&nbsp;META">Section&nbsp;9.2.2, &#8220;META&#8221;</a> for more information on META.</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2475e5366" href="#d2475e5366" class="para">21</a>] </sup>J Mohamed Zahoor goes into some more detail on the Master Architecture in this blog posting, <a class="link" href="http://blog.zahoor.in/2012/08/hbase-hmaster-architecture/" target="_top">HBase HMaster Architecture
+             </p></div><div class="section" title="9.5.4.2.&nbsp;CatalogJanitor"><div class="titlepage"><div><div><h4 class="title"><a name="master.processes.catalog"></a>9.5.4.2.&nbsp;CatalogJanitor</h4></div></div></div><p>Periodically checks and cleans up the .META. table.  See <a class="xref" href="arch.catalog.html#arch.catalog.meta" title="9.2.2.&nbsp;META">Section&nbsp;9.2.2, &#8220;META&#8221;</a> for more information on META.</p></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2519e5599" href="#d2519e5599" class="para">22</a>] </sup>J Mohamed Zahoor goes into some more detail on the Master Architecture in this blog posting, <a class="link" href="http://blog.zahoor.in/2012/08/hbase-hmaster-architecture/" target="_top">HBase HMaster Architecture
             </a>.</p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
     var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
     var disqus_url = 'http://hbase.apache.org/book';

Modified: hbase/hbase.apache.org/trunk/book/ops.monitoring.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/ops.monitoring.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/ops.monitoring.html (original)
+++ hbase/hbase.apache.org/trunk/book/ops.monitoring.html Sat Apr  6 06:08:56 2013
@@ -14,13 +14,13 @@
       </p><p>
       For more information on HBase metrics, see <a class="xref" href="hbase_metrics.html" title="14.4.&nbsp;HBase Metrics">Section&nbsp;14.4, &#8220;HBase Metrics&#8221;</a>.
       </p></div><div class="section" title="14.5.2.&nbsp;Slow Query Log"><div class="titlepage"><div><div><h3 class="title"><a name="ops.slow.query"></a>14.5.2.&nbsp;Slow Query Log</h3></div></div></div><p>The HBase slow query log consists of parseable JSON structures describing the properties of those client operations (Gets, Puts, Deletes, etc.) that either took too long to run, or produced too much output. The thresholds for "too long to run" and "too much output" are configurable, as described below. The output is produced inline in the main region server logs so that it is easy to discover further details from context with other logged events. It is also prepended with identifying tags <code class="constant">(responseTooSlow)</code>, <code class="constant">(responseTooLarge)</code>, <code class="constant">(operationTooSlow)</code>, and <code class="constant">(operationTooLarge)</code> in order to enable easy filtering with grep, in case the user desires to see only slow
  queries.
-</p><div class="section" title="14.5.2.1.&nbsp;Configuration"><div class="titlepage"><div><div><h4 class="title"><a name="d2475e9788"></a>14.5.2.1.&nbsp;Configuration</h4></div></div></div><p>There are two configuration knobs that can be used to adjust the thresholds for when queries are logged.
+</p><div class="section" title="14.5.2.1.&nbsp;Configuration"><div class="titlepage"><div><div><h4 class="title"><a name="d2519e9969"></a>14.5.2.1.&nbsp;Configuration</h4></div></div></div><p>There are two configuration knobs that can be used to adjust the thresholds for when queries are logged.
 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><code class="varname">hbase.ipc.warn.response.time</code> Maximum number of milliseconds that a query can be run without being logged. Defaults to 10000, or 10 seconds. Can be set to -1 to disable logging by time.
 </li><li class="listitem"><code class="varname">hbase.ipc.warn.response.size</code> Maximum byte size of response that a query can return without being logged. Defaults to 100 megabytes. Can be set to -1 to disable logging by size.
-</li></ul></div></div><div class="section" title="14.5.2.2.&nbsp;Metrics"><div class="titlepage"><div><div><h4 class="title"><a name="d2475e9802"></a>14.5.2.2.&nbsp;Metrics</h4></div></div></div><p>The slow query log exposes to metrics to JMX.
+</li></ul></div></div><div class="section" title="14.5.2.2.&nbsp;Metrics"><div class="titlepage"><div><div><h4 class="title"><a name="d2519e9983"></a>14.5.2.2.&nbsp;Metrics</h4></div></div></div><p>The slow query log exposes to metrics to JMX.
 </p><div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem"><code class="varname">hadoop.regionserver_rpc_slowResponse</code> a global metric reflecting the durations of all responses that triggered logging.</li><li class="listitem"><code class="varname">hadoop.regionserver_rpc_methodName.aboveOneSec</code> A metric reflecting the durations of all responses that lasted for more than one second.</li></ul></div><p>
-</p></div><div class="section" title="14.5.2.3.&nbsp;Output"><div class="titlepage"><div><div><h4 class="title"><a name="d2475e9817"></a>14.5.2.3.&nbsp;Output</h4></div></div></div><p>The output is tagged with operation e.g. <code class="constant">(operationTooSlow)</code> if the call was a client operation, such as a Put, Get, or Delete, which we expose detailed fingerprint information for. If not, it is tagged <code class="constant">(responseTooSlow)</code> and still produces parseable JSON output, but with less verbose information solely regarding its duration and size in the RPC itself. <code class="constant">TooLarge</code> is substituted for <code class="constant">TooSlow</code> if the response size triggered the logging, with <code class="constant">TooLarge</code> appearing even in the case that both size and duration triggered logging.
-</p></div><div class="section" title="14.5.2.4.&nbsp;Example"><div class="titlepage"><div><div><h4 class="title"><a name="d2475e9837"></a>14.5.2.4.&nbsp;Example</h4></div></div></div><p>
+</p></div><div class="section" title="14.5.2.3.&nbsp;Output"><div class="titlepage"><div><div><h4 class="title"><a name="d2519e9998"></a>14.5.2.3.&nbsp;Output</h4></div></div></div><p>The output is tagged with operation e.g. <code class="constant">(operationTooSlow)</code> if the call was a client operation, such as a Put, Get, or Delete, which we expose detailed fingerprint information for. If not, it is tagged <code class="constant">(responseTooSlow)</code> and still produces parseable JSON output, but with less verbose information solely regarding its duration and size in the RPC itself. <code class="constant">TooLarge</code> is substituted for <code class="constant">TooSlow</code> if the response size triggered the logging, with <code class="constant">TooLarge</code> appearing even in the case that both size and duration triggered logging.
+</p></div><div class="section" title="14.5.2.4.&nbsp;Example"><div class="titlepage"><div><div><h4 class="title"><a name="d2519e10018"></a>14.5.2.4.&nbsp;Example</h4></div></div></div><p>
 </p><pre class="programlisting">2011-09-08 10:01:25,824 WARN org.apache.hadoop.ipc.HBaseServer: (operationTooSlow): {"tables":{"riley2":{"puts":[{"totalColumns":11,"families":{"actions":[{"timestamp":1315501284459,"qualifier":"0","vlen":9667580},{"timestamp":1315501284459,"qualifier":"1","vlen":10122412},{"timestamp":1315501284459,"qualifier":"2","vlen":11104617},{"timestamp":1315501284459,"qualifier":"3","vlen":13430635}]},"row":"cfcd208495d565ef66e7dff9f98764da:0"}],"families":["actions"]}},"processingtimems":956,"client":"10.47.34.63:33623","starttimems":1315501284456,"queuetimems":0,"totalPuts":1,"class":"HRegionServer","responsesize":0,"method":"multiPut"}</pre><p>
 </p><p>Note that everything inside the "tables" structure is output produced by MultiPut's fingerprint, while the rest of the information is RPC-specific, such as processing time and client IP/port. Other client operations follow the same pattern and the same general structure, with necessary differences due to the nature of the individual operations. In the case that the call is not a client operation, that detailed fingerprint information will be completely absent.
 </p><p>This particular example, for example, would indicate that the likely cause of slowness is simply a very large (on the order of 100MB) multiput, as we can tell by the "vlen," or value length, fields of each put in the multiPut.

Modified: hbase/hbase.apache.org/trunk/book/perf.hdfs.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/perf.hdfs.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/perf.hdfs.html (original)
+++ hbase/hbase.apache.org/trunk/book/perf.hdfs.html Sat Apr  6 06:08:56 2013
@@ -13,7 +13,7 @@ read directly from disk instead of going
 data is local. What this means for HBase is that the RegionServers can
 read directly off their machine's disks instead of having to open a
 socket to talk to the DataNode, the former being generally much
-faster<sup>[<a name="d2475e7777" href="#ftn.d2475e7777" class="footnote">27</a>]</sup>.
+faster<sup>[<a name="d2519e8010" href="#ftn.d2519e8010" class="footnote">28</a>]</sup>.
 Also see <a class="link" href="http://search-hadoop.com/m/zV6dKrLCVh1" target="_top">HBase, mail # dev - read short circuit</a> thread for
 more discussion around short circuit reads.
 </p><p>To enable "short circuit" reads, you must set two configurations.
@@ -37,7 +37,7 @@ the data will still be read.
      returning the most current row or specified timestamps, etc.), and as such HBase is 4-5 times slower than HDFS in this
      processing context.  Not that there isn't room for improvement (and this gap will, over time, be reduced), but HDFS
       will always be faster in this use-case.
-     </p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2475e7777" href="#d2475e7777" class="para">27</a>] </sup>See JD's <a class="link" href="http://files.meetup.com/1350427/hug_ebay_jdcryans.pdf" target="_top">Performance Talk</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
+     </p></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2519e8010" href="#d2519e8010" class="para">28</a>] </sup>See JD's <a class="link" href="http://files.meetup.com/1350427/hug_ebay_jdcryans.pdf" target="_top">Performance Talk</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
     var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
     var disqus_url = 'http://hbase.apache.org/book';
     var disqus_identifier = 'perf.hdfs';

Modified: hbase/hbase.apache.org/trunk/book/perf.reading.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/perf.reading.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/perf.reading.html (original)
+++ hbase/hbase.apache.org/trunk/book/perf.reading.html Sat Apr  6 06:08:56 2013
@@ -53,34 +53,34 @@ htable.close();</pre></div><div class="s
     Table Creation: Pre-Creating Regions
     &#8221;</a>, as well as <a class="xref" href="perf.configurations.html" title="11.4.&nbsp;HBase Configurations">Section&nbsp;11.4, &#8220;HBase Configurations&#8221;</a> </p></div><div class="section" title="11.9.8.&nbsp;Bloom Filters"><div class="titlepage"><div><div><h3 class="title"><a name="blooms"></a>11.9.8.&nbsp;Bloom Filters</h3></div></div></div><p>Enabling Bloom Filters can save your having to go to disk and
          can help improve read latencys.</p><p><a class="link" href="http://en.wikipedia.org/wiki/Bloom_filter" target="_top">Bloom filters</a> were developed over in <a class="link" href="https://issues.apache.org/jira/browse/HBASE-1200" target="_top">HBase-1200
-    Add bloomfilters</a>.<sup>[<a name="d2475e7602" href="#ftn.d2475e7602" class="footnote">25</a>]</sup><sup>[<a name="d2475e7614" href="#ftn.d2475e7614" class="footnote">26</a>]</sup></p><p>See also <a class="xref" href="perf.schema.html#schema.bloom" title="11.6.4.&nbsp;Bloom Filters">Section&nbsp;11.6.4, &#8220;Bloom Filters&#8221;</a>.
+    Add bloomfilters</a>.<sup>[<a name="d2519e7835" href="#ftn.d2519e7835" class="footnote">26</a>]</sup><sup>[<a name="d2519e7847" href="#ftn.d2519e7847" class="footnote">27</a>]</sup></p><p>See also <a class="xref" href="perf.schema.html#schema.bloom" title="11.6.4.&nbsp;Bloom Filters">Section&nbsp;11.6.4, &#8220;Bloom Filters&#8221;</a>.
         </p><div class="section" title="11.9.8.1.&nbsp;Bloom StoreFile footprint"><div class="titlepage"><div><div><h4 class="title"><a name="bloom_footprint"></a>11.9.8.1.&nbsp;Bloom StoreFile footprint</h4></div></div></div><p>Bloom filters add an entry to the <code class="classname">StoreFile</code>
       general <code class="classname">FileInfo</code> data structure and then two
       extra entries to the <code class="classname">StoreFile</code> metadata
-      section.</p><div class="section" title="11.9.8.1.1.&nbsp;BloomFilter in the StoreFile FileInfo data structure"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e7638"></a>11.9.8.1.1.&nbsp;BloomFilter in the <code class="classname">StoreFile</code>
+      section.</p><div class="section" title="11.9.8.1.1.&nbsp;BloomFilter in the StoreFile FileInfo data structure"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e7871"></a>11.9.8.1.1.&nbsp;BloomFilter in the <code class="classname">StoreFile</code>
         <code class="classname">FileInfo</code> data structure</h5></div></div></div><p><code class="classname">FileInfo</code> has a
           <code class="varname">BLOOM_FILTER_TYPE</code> entry which is set to
           <code class="varname">NONE</code>, <code class="varname">ROW</code> or
-          <code class="varname">ROWCOL.</code></p></div><div class="section" title="11.9.8.1.2.&nbsp;BloomFilter entries in StoreFile metadata"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e7662"></a>11.9.8.1.2.&nbsp;BloomFilter entries in <code class="classname">StoreFile</code>
+          <code class="varname">ROWCOL.</code></p></div><div class="section" title="11.9.8.1.2.&nbsp;BloomFilter entries in StoreFile metadata"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e7895"></a>11.9.8.1.2.&nbsp;BloomFilter entries in <code class="classname">StoreFile</code>
         metadata</h5></div></div></div><p><code class="varname">BLOOM_FILTER_META</code> holds Bloom Size, Hash
           Function used, etc. Its small in size and is cached on
           <code class="classname">StoreFile.Reader</code> load</p><p><code class="varname">BLOOM_FILTER_DATA</code> is the actual bloomfilter
           data. Obtained on-demand. Stored in the LRU cache, if it is enabled
-          (Its enabled by default).</p></div></div><div class="section" title="11.9.8.2.&nbsp;Bloom Filter Configuration"><div class="titlepage"><div><div><h4 class="title"><a name="config.bloom"></a>11.9.8.2.&nbsp;Bloom Filter Configuration</h4></div></div></div><div class="section" title="11.9.8.2.1.&nbsp;io.hfile.bloom.enabled global kill switch"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e7682"></a>11.9.8.2.1.&nbsp;<code class="varname">io.hfile.bloom.enabled</code> global kill
+          (Its enabled by default).</p></div></div><div class="section" title="11.9.8.2.&nbsp;Bloom Filter Configuration"><div class="titlepage"><div><div><h4 class="title"><a name="config.bloom"></a>11.9.8.2.&nbsp;Bloom Filter Configuration</h4></div></div></div><div class="section" title="11.9.8.2.1.&nbsp;io.hfile.bloom.enabled global kill switch"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e7915"></a>11.9.8.2.1.&nbsp;<code class="varname">io.hfile.bloom.enabled</code> global kill
         switch</h5></div></div></div><p><code class="code">io.hfile.bloom.enabled</code> in
         <code class="classname">Configuration</code> serves as the kill switch in case
-        something goes wrong. Default = <code class="varname">true</code>.</p></div><div class="section" title="11.9.8.2.2.&nbsp;io.hfile.bloom.error.rate"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e7697"></a>11.9.8.2.2.&nbsp;<code class="varname">io.hfile.bloom.error.rate</code></h5></div></div></div><p><code class="varname">io.hfile.bloom.error.rate</code> = average false
+        something goes wrong. Default = <code class="varname">true</code>.</p></div><div class="section" title="11.9.8.2.2.&nbsp;io.hfile.bloom.error.rate"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e7930"></a>11.9.8.2.2.&nbsp;<code class="varname">io.hfile.bloom.error.rate</code></h5></div></div></div><p><code class="varname">io.hfile.bloom.error.rate</code> = average false
         positive rate. Default = 1%. Decrease rate by &frac12; (e.g. to .5%) == +1
-        bit per bloom entry.</p></div><div class="section" title="11.9.8.2.3.&nbsp;io.hfile.bloom.max.fold"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e7705"></a>11.9.8.2.3.&nbsp;<code class="varname">io.hfile.bloom.max.fold</code></h5></div></div></div><p><code class="varname">io.hfile.bloom.max.fold</code> = guaranteed minimum
+        bit per bloom entry.</p></div><div class="section" title="11.9.8.2.3.&nbsp;io.hfile.bloom.max.fold"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e7938"></a>11.9.8.2.3.&nbsp;<code class="varname">io.hfile.bloom.max.fold</code></h5></div></div></div><p><code class="varname">io.hfile.bloom.max.fold</code> = guaranteed minimum
         fold rate. Most people should leave this alone. Default = 7, or can
         collapse to at least 1/128th of original size. See the
         <span class="emphasis"><em>Development Process</em></span> section of the document <a class="link" href="https://issues.apache.org/jira/secure/attachment/12444007/Bloom_Filters_in_HBase.pdf" target="_top">BloomFilters
-        in HBase</a> for more on what this option means.</p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2475e7602" href="#d2475e7602" class="para">25</a>] </sup>For description of the development process -- why static blooms
+        in HBase</a> for more on what this option means.</p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2519e7835" href="#d2519e7835" class="para">26</a>] </sup>For description of the development process -- why static blooms
         rather than dynamic -- and for an overview of the unique properties
         that pertain to blooms in HBase, as well as possible future
         directions, see the <span class="emphasis"><em>Development Process</em></span> section
         of the document <a class="link" href="https://issues.apache.org/jira/secure/attachment/12444007/Bloom_Filters_in_HBase.pdf" target="_top">BloomFilters
-        in HBase</a> attached to <a class="link" href="https://issues.apache.org/jira/browse/HBASE-1200" target="_top">HBase-1200</a>.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2475e7614" href="#d2475e7614" class="para">26</a>] </sup>The bloom filters described here are actually version two of
+        in HBase</a> attached to <a class="link" href="https://issues.apache.org/jira/browse/HBASE-1200" target="_top">HBase-1200</a>.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2519e7847" href="#d2519e7847" class="para">27</a>] </sup>The bloom filters described here are actually version two of
         blooms in HBase. In versions up to 0.19.x, HBase had a dynamic bloom
         option based on work done by the <a class="link" href="http://www.one-lab.org" target="_top">European Commission One-Lab
         Project 034819</a>. The core of the HBase bloom work was later

Modified: hbase/hbase.apache.org/trunk/book/physical.view.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/physical.view.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/physical.view.html (original)
+++ hbase/hbase.apache.org/trunk/book/physical.view.html Sat Apr  6 06:08:56 2013
@@ -5,8 +5,8 @@
         Physically they are stored on a per-column family basis.  New columns
         (i.e., <code class="varname">columnfamily:column</code>) can be added to any
         column family without pre-announcing them.
-        </p><div class="table"><a name="d2475e3130"></a><p class="title"><b>Table&nbsp;5.2.&nbsp;ColumnFamily <code class="varname">anchor</code></b></p><div class="table-contents"><table summary="ColumnFamily anchor" border="1"><colgroup><col align="left" class="c1"><col align="left" class="c2"><col align="left" class="c3"></colgroup><thead><tr><th align="left">Row Key</th><th align="left">Time Stamp</th><th align="left">Column Family <code class="varname">anchor</code></th></tr></thead><tbody><tr><td align="left">"com.cnn.www"</td><td align="left">t9</td><td align="left"><code class="varname">anchor:cnnsi.com</code> = "CNN"</td></tr><tr><td align="left">"com.cnn.www"</td><td align="left">t8</td><td align="left"><code class="varname">anchor:my.look.ca</code> = "CNN.com"</td></tr></tbody></table></div></div><p><br class="table-break">
-    </p><div class="table"><a name="d2475e3169"></a><p class="title"><b>Table&nbsp;5.3.&nbsp;ColumnFamily <code class="varname">contents</code></b></p><div class="table-contents"><table summary="ColumnFamily contents" border="1"><colgroup><col align="left" class="c1"><col align="left" class="c2"><col align="left" class="c3"></colgroup><thead><tr><th align="left">Row Key</th><th align="left">Time Stamp</th><th align="left">ColumnFamily "contents:"</th></tr></thead><tbody><tr><td align="left">"com.cnn.www"</td><td align="left">t6</td><td align="left"><code class="varname">contents:html</code> = "&lt;html&gt;..."</td></tr><tr><td align="left">"com.cnn.www"</td><td align="left">t5</td><td align="left"><code class="varname">contents:html</code> = "&lt;html&gt;..."</td></tr><tr><td align="left">"com.cnn.www"</td><td align="left">t3</td><td align="left"><code class="varname">contents:html</code> = "&lt;html&gt;..."</td></tr></tbody></table></div></div><p><br class="table-break">
+        </p><div class="table"><a name="d2519e3153"></a><p class="title"><b>Table&nbsp;5.2.&nbsp;ColumnFamily <code class="varname">anchor</code></b></p><div class="table-contents"><table summary="ColumnFamily anchor" border="1"><colgroup><col align="left" class="c1"><col align="left" class="c2"><col align="left" class="c3"></colgroup><thead><tr><th align="left">Row Key</th><th align="left">Time Stamp</th><th align="left">Column Family <code class="varname">anchor</code></th></tr></thead><tbody><tr><td align="left">"com.cnn.www"</td><td align="left">t9</td><td align="left"><code class="varname">anchor:cnnsi.com</code> = "CNN"</td></tr><tr><td align="left">"com.cnn.www"</td><td align="left">t8</td><td align="left"><code class="varname">anchor:my.look.ca</code> = "CNN.com"</td></tr></tbody></table></div></div><p><br class="table-break">
+    </p><div class="table"><a name="d2519e3192"></a><p class="title"><b>Table&nbsp;5.3.&nbsp;ColumnFamily <code class="varname">contents</code></b></p><div class="table-contents"><table summary="ColumnFamily contents" border="1"><colgroup><col align="left" class="c1"><col align="left" class="c2"><col align="left" class="c3"></colgroup><thead><tr><th align="left">Row Key</th><th align="left">Time Stamp</th><th align="left">ColumnFamily "contents:"</th></tr></thead><tbody><tr><td align="left">"com.cnn.www"</td><td align="left">t6</td><td align="left"><code class="varname">contents:html</code> = "&lt;html&gt;..."</td></tr><tr><td align="left">"com.cnn.www"</td><td align="left">t5</td><td align="left"><code class="varname">contents:html</code> = "&lt;html&gt;..."</td></tr><tr><td align="left">"com.cnn.www"</td><td align="left">t3</td><td align="left"><code class="varname">contents:html</code> = "&lt;html&gt;..."</td></tr></tbody></table></div></div><p><br class="table-break">
     It is important to note in the diagram above that the empty cells shown in the
     conceptual view are not stored since they need not be in a column-oriented
     storage format. Thus a request for the value of the <code class="varname">contents:html</code>

Modified: hbase/hbase.apache.org/trunk/book/quickstart.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/quickstart.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/quickstart.html (original)
+++ hbase/hbase.apache.org/trunk/book/quickstart.html Sat Apr  6 06:08:56 2013
@@ -11,7 +11,7 @@
             127.0.0.1 localhost
             127.0.0.1 ubuntu.ubuntu-domain ubuntu
 </pre><p>
-        </p></div><div class="section" title="1.2.1.&nbsp;Download and unpack the latest stable release."><div class="titlepage"><div><div><h3 class="title"><a name="d2475e105"></a>1.2.1.&nbsp;Download and unpack the latest stable release.</h3></div></div></div><p>Choose a download site from this list of <a class="link" href="http://www.apache.org/dyn/closer.cgi/hbase/" target="_top">Apache Download
+        </p></div><div class="section" title="1.2.1.&nbsp;Download and unpack the latest stable release."><div class="titlepage"><div><div><h3 class="title"><a name="d2519e105"></a>1.2.1.&nbsp;Download and unpack the latest stable release.</h3></div></div></div><p>Choose a download site from this list of <a class="link" href="http://www.apache.org/dyn/closer.cgi/hbase/" target="_top">Apache Download
       Mirrors</a>. Click on the suggested top link. This will take you to a
       mirror of <span class="emphasis"><em>HBase Releases</em></span>. Click on the folder named
       <code class="filename">stable</code> and then download the file that ends in
@@ -93,7 +93,7 @@ cf:a        timestamp=1288380727188, val
 0 row(s) in 1.0930 seconds
 hbase(main):013:0&gt; drop 'test'
 0 row(s) in 0.0770 seconds </pre><p>Exit the shell by typing exit.</p><pre class="programlisting">hbase(main):014:0&gt; exit</pre></div><div class="section" title="1.2.4.&nbsp;Stopping HBase"><div class="titlepage"><div><div><h3 class="title"><a name="stopping"></a>1.2.4.&nbsp;Stopping HBase</h3></div></div></div><p>Stop your hbase instance by running the stop script.</p><pre class="programlisting">$ ./bin/stop-hbase.sh
-stopping hbase...............</pre></div><div class="section" title="1.2.5.&nbsp;Where to go next"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e265"></a>1.2.5.&nbsp;Where to go next</h3></div></div></div><p>The above described standalone setup is good for testing and
+stopping hbase...............</pre></div><div class="section" title="1.2.5.&nbsp;Where to go next"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e265"></a>1.2.5.&nbsp;Where to go next</h3></div></div></div><p>The above described standalone setup is good for testing and
           experiments only. In the next chapter, <a class="xref" href="configuration.html" title="Chapter&nbsp;2.&nbsp;Apache HBase (TM) Configuration">Chapter&nbsp;2, <i>Apache HBase (TM) Configuration</i></a>,
       we'll go into depth on the different HBase run modes, system requirements
       running HBase, and critical configurations setting up a distributed HBase deploy.</p></div></div><div id="disqus_thread"></div><script type="text/javascript">

Modified: hbase/hbase.apache.org/trunk/book/regions.arch.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/regions.arch.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/regions.arch.html (original)
+++ hbase/hbase.apache.org/trunk/book/regions.arch.html Sat Apr  6 06:08:56 2013
@@ -57,22 +57,22 @@
           to the RegionServer.
         </p><p>For more information, see <a class="link" href="http://hadoop.apache.org/common/docs/r0.20.205.0/hdfs_design.html#Replica+Placement%3A+The+First+Baby+Steps" target="_top">HDFS Design on Replica Placement</a>
         and also Lars George's blog on <a class="link" href="http://www.larsgeorge.com/2010/05/hbase-file-locality-in-hdfs.html" target="_top">HBase and HDFS locality</a>.
-        </p></div><div class="section" title="9.7.4.&nbsp;Region Splits"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e5763"></a>9.7.4.&nbsp;Region Splits</h3></div></div></div><p>Splits run unaided on the RegionServer; i.e. the Master does not
+        </p></div><div class="section" title="9.7.4.&nbsp;Region Splits"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e5996"></a>9.7.4.&nbsp;Region Splits</h3></div></div></div><p>Splits run unaided on the RegionServer; i.e. the Master does not
         participate. The RegionServer splits a region, offlines the split
         region and then adds the daughter regions to META, opens daughters on
         the parent's hosting RegionServer and then reports the split to the
         Master. See <a class="xref" href="important_configurations.html#disable.splitting" title="2.5.2.7.&nbsp;Managed Splitting">Section&nbsp;2.5.2.7, &#8220;Managed Splitting&#8221;</a> for how to manually manage
-        splits (and for why you might do this)</p><div class="section" title="9.7.4.1.&nbsp;Custom Split Policies"><div class="titlepage"><div><div><h4 class="title"><a name="d2475e5770"></a>9.7.4.1.&nbsp;Custom Split Policies</h4></div></div></div><p>The default split policy can be overwritten using a custom <a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.html" target="_top">RegionSplitPolicy</a> (HBase 0.94+).
+        splits (and for why you might do this)</p><div class="section" title="9.7.4.1.&nbsp;Custom Split Policies"><div class="titlepage"><div><div><h4 class="title"><a name="d2519e6003"></a>9.7.4.1.&nbsp;Custom Split Policies</h4></div></div></div><p>The default split policy can be overwritten using a custom <a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.html" target="_top">RegionSplitPolicy</a> (HBase 0.94+).
           Typically a custom split policy should extend HBase's default split policy: <a class="link" href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html" target="_top">ConstantSizeRegionSplitPolicy</a>.
           </p><p>The policy can set globally through the HBaseConfiguration used or on a per table basis:
 </p><pre class="programlisting">
 HTableDescriptor myHtd = ...;
 myHtd.setValue(HTableDescriptor.SPLIT_POLICY, MyCustomSplitPolicy.class.getName());
 </pre><p>
-          </p></div></div><div class="section" title="9.7.5.&nbsp;Online Region Merges"><div class="titlepage"><div><div><h3 class="title"><a name="d2475e5786"></a>9.7.5.&nbsp;Online Region Merges</h3></div></div></div><p>Both Master and Regionserver participate in the event of online region merges.
+          </p></div></div><div class="section" title="9.7.5.&nbsp;Online Region Merges"><div class="titlepage"><div><div><h3 class="title"><a name="d2519e6019"></a>9.7.5.&nbsp;Online Region Merges</h3></div></div></div><p>Both Master and Regionserver participate in the event of online region merges.
         Client sends merge RPC to master, then master moves the regions together to the
         same regionserver where the more heavily loaded region resided, finally master
-        send merge request to this regionserver and regionserver run the region merges. 
+        send merge request to this regionserver and regionserver run the region merges.
         Similar with process of region splits, region merges run as a local transaction
         on the regionserver, offlines the regions and then merges two regions on the file
         system, atomically delete merging regions from META and add merged region to the META,
@@ -82,14 +82,14 @@ myHtd.setValue(HTableDescriptor.SPLIT_PO
           hbase&gt; merge_region 'ENCODED_REGIONNAME', 'ENCODED_REGIONNAME', true
           </pre><p>
           It's an asynchronous operation and call returns immediately without waiting merge completed.
-          Passing 'true' as the optional third parameter will force a merge ('force' merges regardless 
+          Passing 'true' as the optional third parameter will force a merge ('force' merges regardless
           else merge will fail unless passed adjacent regions. 'force' is for expert use only)
         </p></div><div class="section" title="9.7.6.&nbsp;Store"><div class="titlepage"><div><div><h3 class="title"><a name="store"></a>9.7.6.&nbsp;Store</h3></div></div></div><p>A Store hosts a MemStore and 0 or more StoreFiles (HFiles). A Store corresponds to a column family for a table for a given region.
           </p><div class="section" title="9.7.6.1.&nbsp;MemStore"><div class="titlepage"><div><div><h4 class="title"><a name="store.memstore"></a>9.7.6.1.&nbsp;MemStore</h4></div></div></div><p>The MemStore holds in-memory modifications to the Store.  Modifications are KeyValues.
        When asked to flush, current memstore is moved to snapshot and is cleared.
        HBase continues to serve edits out of new memstore and backing snapshot until flusher reports in that the
        flush succeeded. At this point the snapshot is let go.</p></div><div class="section" title="9.7.6.2.&nbsp;StoreFile (HFile)"><div class="titlepage"><div><div><h4 class="title"><a name="hfile"></a>9.7.6.2.&nbsp;StoreFile (HFile)</h4></div></div></div><p>StoreFiles are where your data lives.
-      </p><div class="section" title="9.7.6.2.1.&nbsp;HFile Format"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e5811"></a>9.7.6.2.1.&nbsp;HFile Format</h5></div></div></div><p>The <span class="emphasis"><em>hfile</em></span> file format is based on
+      </p><div class="section" title="9.7.6.2.1.&nbsp;HFile Format"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e6044"></a>9.7.6.2.1.&nbsp;HFile Format</h5></div></div></div><p>The <span class="emphasis"><em>hfile</em></span> file format is based on
               the SSTable file described in the <a class="link" href="http://research.google.com/archive/bigtable.html" target="_top">BigTable [2006]</a> paper and on
               Hadoop's <a class="link" href="http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/io/file/tfile/TFile.html" target="_top">tfile</a>
               (The unit test suite and the compression harness were taken directly from tfile).

Modified: hbase/hbase.apache.org/trunk/book/regionserver.arch.html
URL: http://svn.apache.org/viewvc/hbase/hbase.apache.org/trunk/book/regionserver.arch.html?rev=1465200&r1=1465199&r2=1465200&view=diff
==============================================================================
--- hbase/hbase.apache.org/trunk/book/regionserver.arch.html (original)
+++ hbase/hbase.apache.org/trunk/book/regionserver.arch.html Sat Apr  6 06:08:56 2013
@@ -61,22 +61,22 @@
         For more general information about the concept of write ahead logs, see the Wikipedia
         <a class="link" href="http://en.wikipedia.org/wiki/Write-ahead_logging" target="_top">Write-Ahead Log</a> article.
        </p></div><div class="section" title="9.6.5.2.&nbsp;WAL Flushing"><div class="titlepage"><div><div><h4 class="title"><a name="wal_flush"></a>9.6.5.2.&nbsp;WAL Flushing</h4></div></div></div><p>TODO (describe).
-          </p></div><div class="section" title="9.6.5.3.&nbsp;WAL Splitting"><div class="titlepage"><div><div><h4 class="title"><a name="wal_splitting"></a>9.6.5.3.&nbsp;WAL Splitting</h4></div></div></div><div class="section" title="9.6.5.3.1.&nbsp;How edits are recovered from a crashed RegionServer"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e5592"></a>9.6.5.3.1.&nbsp;How edits are recovered from a crashed RegionServer</h5></div></div></div><p>When a RegionServer crashes, it will lose its ephemeral lease in
-         ZooKeeper...TODO</p></div><div class="section" title="9.6.5.3.2.&nbsp;hbase.hlog.split.skip.errors"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e5597"></a>9.6.5.3.2.&nbsp;<code class="varname">hbase.hlog.split.skip.errors</code></h5></div></div></div><p>When set to <code class="constant">true</code>, any error
+          </p></div><div class="section" title="9.6.5.3.&nbsp;WAL Splitting"><div class="titlepage"><div><div><h4 class="title"><a name="wal_splitting"></a>9.6.5.3.&nbsp;WAL Splitting</h4></div></div></div><div class="section" title="9.6.5.3.1.&nbsp;How edits are recovered from a crashed RegionServer"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e5825"></a>9.6.5.3.1.&nbsp;How edits are recovered from a crashed RegionServer</h5></div></div></div><p>When a RegionServer crashes, it will lose its ephemeral lease in
+         ZooKeeper...TODO</p></div><div class="section" title="9.6.5.3.2.&nbsp;hbase.hlog.split.skip.errors"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e5830"></a>9.6.5.3.2.&nbsp;<code class="varname">hbase.hlog.split.skip.errors</code></h5></div></div></div><p>When set to <code class="constant">true</code>, any error
         encountered splitting will be logged, the problematic WAL will be
         moved into the <code class="filename">.corrupt</code> directory under the hbase
         <code class="varname">rootdir</code>, and processing will continue. If set to
         <code class="constant">false</code>, the default, the exception will be propagated and the
-        split logged as failed.<sup>[<a name="d2475e5615" href="#ftn.d2475e5615" class="footnote">22</a>]</sup></p></div><div class="section" title="9.6.5.3.3.&nbsp;How EOFExceptions are treated when splitting a crashed RegionServers' WALs"><div class="titlepage"><div><div><h5 class="title"><a name="d2475e5621"></a>9.6.5.3.3.&nbsp;How EOFExceptions are treated when splitting a crashed
+        split logged as failed.<sup>[<a name="d2519e5848" href="#ftn.d2519e5848" class="footnote">23</a>]</sup></p></div><div class="section" title="9.6.5.3.3.&nbsp;How EOFExceptions are treated when splitting a crashed RegionServers' WALs"><div class="titlepage"><div><div><h5 class="title"><a name="d2519e5854"></a>9.6.5.3.3.&nbsp;How EOFExceptions are treated when splitting a crashed
         RegionServers' WALs</h5></div></div></div><p>If we get an EOF while splitting logs, we proceed with the split
         even when <code class="varname">hbase.hlog.split.skip.errors</code> ==
         <code class="constant">false</code>. An EOF while reading the last log in the
         set of files to split is near-guaranteed since the RegionServer likely
         crashed mid-write of a record. But we'll continue even if we got an
-        EOF reading other than the last file in the set.<sup>[<a name="d2475e5632" href="#ftn.d2475e5632" class="footnote">23</a>]</sup></p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2475e5615" href="#d2475e5615" class="para">22</a>] </sup>See <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2958" target="_top">HBASE-2958
+        EOF reading other than the last file in the set.<sup>[<a name="d2519e5865" href="#ftn.d2519e5865" class="footnote">24</a>]</sup></p></div></div></div><div class="footnotes"><br><hr width="100" align="left"><div class="footnote"><p><sup>[<a id="ftn.d2519e5848" href="#d2519e5848" class="para">23</a>] </sup>See <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2958" target="_top">HBASE-2958
             When hbase.hlog.split.skip.errors is set to false, we fail the
             split but thats it</a>. We need to do more than just fail split
-            if this flag is set.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2475e5632" href="#d2475e5632" class="para">23</a>] </sup>For background, see <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2643" target="_top">HBASE-2643
+            if this flag is set.</p></div><div class="footnote"><p><sup>[<a id="ftn.d2519e5865" href="#d2519e5865" class="para">24</a>] </sup>For background, see <a class="link" href="https://issues.apache.org/jira/browse/HBASE-2643" target="_top">HBASE-2643
             Figure how to deal with eof splitting logs</a></p></div></div></div><div id="disqus_thread"></div><script type="text/javascript">
     var disqus_shortname = 'hbase'; // required: replace example with your forum shortname
     var disqus_url = 'http://hbase.apache.org/book';



Mime
View raw message