incubator-accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bil...@apache.org
Subject svn commit: r1202418 - in /incubator/accumulo: branches/1.4/docs/src/user_manual/ branches/1.4/docs/src/user_manual/chapters/ site/trunk/content/accumulo/user_manual_1.3-incubating/ site/trunk/content/accumulo/user_manual_1.4-incubating/ trunk/docs/src...
Date Tue, 15 Nov 2011 20:53:03 GMT
Author: billie
Date: Tue Nov 15 20:53:02 2011
New Revision: 1202418

URL: http://svn.apache.org/viewvc?rev=1202418&view=rev
Log:
ACCUMULO-70 updated site based on user manual changes

Added:
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img6.png   (with
props)
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img7.png   (with
props)
Modified:
    incubator/accumulo/branches/1.4/docs/src/user_manual/   (props changed)
    incubator/accumulo/branches/1.4/docs/src/user_manual/chapters/table_configuration.tex
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Contents.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Shell_Commands.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Table_Configuration.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Administration.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Contents.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Shell_Commands.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Configuration.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Design.mdtext
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img2.png
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img3.png
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img4.png
    incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img5.png
    incubator/accumulo/trunk/docs/src/user_manual/   (props changed)
    incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex

Propchange: incubator/accumulo/branches/1.4/docs/src/user_manual/
------------------------------------------------------------------------------
--- svn:ignore (original)
+++ svn:ignore Tue Nov 15 20:53:02 2011
@@ -1 +1,2 @@
 accumulo_user_manual.toc
+accumulo_user_manual.out

Modified: incubator/accumulo/branches/1.4/docs/src/user_manual/chapters/table_configuration.tex
URL: http://svn.apache.org/viewvc/incubator/accumulo/branches/1.4/docs/src/user_manual/chapters/table_configuration.tex?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/branches/1.4/docs/src/user_manual/chapters/table_configuration.tex
(original)
+++ incubator/accumulo/branches/1.4/docs/src/user_manual/chapters/table_configuration.tex
Tue Nov 15 20:53:02 2011
@@ -456,8 +456,8 @@ never need to occur and major compaction
 configure the file max and compaction ratio such that only merging minor
 compactions occur and major compactions never occur. This should be avoided
 because doing only merging minor compactions causes $O(N^2)$ work to be done.
-The amount of work done by major compactions is $O(N*\log_R(N))$ where $R$ is
-the compaction ratio.
+The amount of work done by major compactions is $O(N*\log_R(N))$ where 
+\textit{R} is the compaction ratio.
 
 Compactions can be initiated manually for a table.  To initiate a minor
 compaction, use the flush command in the shell.  To initiate a major compaction,

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Contents.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Contents.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Contents.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Contents.mdtext
Tue Nov 15 20:53:02 2011
@@ -77,88 +77,85 @@ Notice:    Licensed to the Apache Softwa
 
         * [Setting Iterators via the Shell][36]
         * [Setting Iterators Programmatically][37]
+        * [Versioning Iterators and Timestamps][38]
+        * [Filtering Iterators][39]
 
-    * [Versioning Iterators and Timestamps][38]
-
-        * [Logical Time][39]
-        * [Deletes][40]
-
-    * [Filtering Iterators][41]
-    * [Aggregating Iterators][42]
+    * [Aggregating Iterators][40]
+    * [Block Cache][41]
 
   
 
-* [Table Design][43]
+* [Table Design][42]
 
-    * [Basic Table][44]
-    * [RowID Design][45]
-    * [Indexing][46]
-    * [Entity-Attribute and Graph Tables][47]
-    * [Document-Partitioned Indexing][48]
+    * [Basic Table][43]
+    * [RowID Design][44]
+    * [Indexing][45]
+    * [Entity-Attribute and Graph Tables][46]
+    * [Document-Partitioned Indexing][47]
 
   
 
-* [High-Speed Ingest][49]
+* [High-Speed Ingest][48]
 
-    * [Pre-Splitting New Tables][50]
-    * [Multiple Ingester Clients][51]
-    * [Bulk Ingest][52]
-    * [MapReduce Ingest][53]
+    * [Pre-Splitting New Tables][49]
+    * [Multiple Ingester Clients][50]
+    * [Bulk Ingest][51]
+    * [MapReduce Ingest][52]
 
   
 
-* [Analytics][54]
+* [Analytics][53]
 
-    * [MapReduce][55]
+    * [MapReduce][54]
 
-        * [Mapper and Reducer classes][56]
-        * [AccumuloInputFormat options][57]
-        * [AccumuloOutputFormat options][58]
+        * [Mapper and Reducer classes][55]
+        * [AccumuloInputFormat options][56]
+        * [AccumuloOutputFormat options][57]
 
-    * [Aggregating Iterators][59]
+    * [Aggregating Iterators][58]
 
-        * [Feature Vectors][60]
+        * [Feature Vectors][59]
 
-    * [Statistical Modeling][61]
+    * [Statistical Modeling][60]
 
   
 
-* [Security][62]
+* [Security][61]
 
-    * [Security Label Expressions][63]
-    * [Security Label Expression Syntax][64]
-    * [Authorization][65]
-    * [Secure Authorizations Handling][66]
-    * [Query Services Layer][67]
+    * [Security Label Expressions][62]
+    * [Security Label Expression Syntax][63]
+    * [Authorization][64]
+    * [Secure Authorizations Handling][65]
+    * [Query Services Layer][66]
 
   
 
-* [Administration][68]
+* [Administration][67]
 
-    * [Hardware][69]
-    * [Network][70]
-    * [Installation][71]
-    * [Dependencies][72]
-    * [Configuration][73]
+    * [Hardware][68]
+    * [Network][69]
+    * [Installation][70]
+    * [Dependencies][71]
+    * [Configuration][72]
 
-        * [Edit conf/accumulo-env.sh][74]
-        * [Cluster Specification][75]
-        * [Accumulo Settings][76]
-        * [Deploy Configuration][77]
+        * [Edit conf/accumulo-env.sh][73]
+        * [Cluster Specification][74]
+        * [Accumulo Settings][75]
+        * [Deploy Configuration][76]
 
-    * [Initialization][78]
-    * [Running][79]
+    * [Initialization][77]
+    * [Running][78]
 
-        * [Starting Accumulo][80]
-        * [Stopping Accumulo][81]
+        * [Starting Accumulo][79]
+        * [Stopping Accumulo][80]
 
-    * [Monitoring][82]
-    * [Logging][83]
-    * [Recovery][84]
+    * [Monitoring][81]
+    * [Logging][82]
+    * [Recovery][83]
 
   
 
-* [Shell Commands][85]
+* [Shell Commands][84]
 
   
 
@@ -200,51 +197,50 @@ Notice:    Licensed to the Apache Softwa
    [36]: Table_Configuration.html#Setting_Iterators_via_the_Shell
    [37]: Table_Configuration.html#Setting_Iterators_Programmatically
    [38]: Table_Configuration.html#Versioning_Iterators_and_Timestamps
-   [39]: Table_Configuration.html#Logical_Time
-   [40]: Table_Configuration.html#Deletes
-   [41]: Table_Configuration.html#Filtering_Iterators
-   [42]: Table_Configuration.html#Aggregating_Iterators
-   [43]: Table_Design.html
-   [44]: Table_Design.html#Basic_Table
-   [45]: Table_Design.html#RowID_Design
-   [46]: Table_Design.html#Indexing
-   [47]: Table_Design.html#Entity-Attribute_and_Graph_Tables
-   [48]: Table_Design.html#Document-Partitioned_Indexing
-   [49]: High_Speed_Ingest.html
-   [50]: High_Speed_Ingest.html#Pre-Splitting_New_Tables
-   [51]: High_Speed_Ingest.html#Multiple_Ingester_Clients
-   [52]: High_Speed_Ingest.html#Bulk_Ingest
-   [53]: High_Speed_Ingest.html#MapReduce_Ingest
-   [54]: Analytics.html
-   [55]: Analytics.html#MapReduce
-   [56]: Analytics.html#Mapper_and_Reducer_classes
-   [57]: Analytics.html#AccumuloInputFormat_options
-   [58]: Analytics.html#AccumuloOutputFormat_options
-   [59]: Analytics.html#Aggregating_Iterators
-   [60]: Analytics.html#Feature_Vectors
-   [61]: Analytics.html#Statistical_Modeling
-   [62]: Security.html
-   [63]: Security.html#Security_Label_Expressions
-   [64]: Security.html#Security_Label_Expression_Syntax
-   [65]: Security.html#Authorization
-   [66]: Security.html#Secure_Authorizations_Handling
-   [67]: Security.html#Query_Services_Layer
-   [68]: Administration.html
-   [69]: Administration.html#Hardware
-   [70]: Administration.html#Network
-   [71]: Administration.html#Installation
-   [72]: Administration.html#Dependencies
-   [73]: Administration.html#Configuration
-   [74]: Administration.html#Edit_conf/accumulo-env.sh
-   [75]: Administration.html#Cluster_Specification
-   [76]: Administration.html#Accumulo_Settings
-   [77]: Administration.html#Deploy_Configuration
-   [78]: Administration.html#Initialization
-   [79]: Administration.html#Running
-   [80]: Administration.html#Starting_Accumulo
-   [81]: Administration.html#Stopping_Accumulo
-   [82]: Administration.html#Monitoring
-   [83]: Administration.html#Logging
-   [84]: Administration.html#Recovery
-   [85]: Shell_Commands.html
+   [39]: Table_Configuration.html#Filtering_Iterators
+   [40]: Table_Configuration.html#Aggregating_Iterators
+   [41]: Table_Configuration.html#Block_Cache
+   [42]: Table_Design.html
+   [43]: Table_Design.html#Basic_Table
+   [44]: Table_Design.html#RowID_Design
+   [45]: Table_Design.html#Indexing
+   [46]: Table_Design.html#Entity-Attribute_and_Graph_Tables
+   [47]: Table_Design.html#Document-Partitioned_Indexing
+   [48]: High_Speed_Ingest.html
+   [49]: High_Speed_Ingest.html#Pre-Splitting_New_Tables
+   [50]: High_Speed_Ingest.html#Multiple_Ingester_Clients
+   [51]: High_Speed_Ingest.html#Bulk_Ingest
+   [52]: High_Speed_Ingest.html#MapReduce_Ingest
+   [53]: Analytics.html
+   [54]: Analytics.html#MapReduce
+   [55]: Analytics.html#Mapper_and_Reducer_classes
+   [56]: Analytics.html#AccumuloInputFormat_options
+   [57]: Analytics.html#AccumuloOutputFormat_options
+   [58]: Analytics.html#Aggregating_Iterators
+   [59]: Analytics.html#Feature_Vectors
+   [60]: Analytics.html#Statistical_Modeling
+   [61]: Security.html
+   [62]: Security.html#Security_Label_Expressions
+   [63]: Security.html#Security_Label_Expression_Syntax
+   [64]: Security.html#Authorization
+   [65]: Security.html#Secure_Authorizations_Handling
+   [66]: Security.html#Query_Services_Layer
+   [67]: Administration.html
+   [68]: Administration.html#Hardware
+   [69]: Administration.html#Network
+   [70]: Administration.html#Installation
+   [71]: Administration.html#Dependencies
+   [72]: Administration.html#Configuration
+   [73]: Administration.html#Edit_conf/accumulo-env.sh
+   [74]: Administration.html#Cluster_Specification
+   [75]: Administration.html#Accumulo_Settings
+   [76]: Administration.html#Deploy_Configuration
+   [77]: Administration.html#Initialization
+   [78]: Administration.html#Running
+   [79]: Administration.html#Starting_Accumulo
+   [80]: Administration.html#Stopping_Accumulo
+   [81]: Administration.html#Monitoring
+   [82]: Administration.html#Logging
+   [83]: Administration.html#Recovery
+   [84]: Shell_Commands.html
 

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Shell_Commands.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Shell_Commands.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Shell_Commands.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Shell_Commands.mdtext
Tue Nov 15 20:53:02 2011
@@ -336,7 +336,7 @@ Notice:    Licensed to the Apache Softwa
   
     usage: listscans [-?] [-np] [-ts <tablet server>]   
     description: list what scans are currently running in accumulo. See the   
-           accumulo.core.client.admin.ActiveScan javadoc for more information   
+           org.apache.accumulo.core.client.admin.ActiveScan javadoc for more information
  
            about columns.   
       -?,-help  display this help   
       -np,-no-pagination  disables pagination of output   

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Table_Configuration.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Table_Configuration.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Table_Configuration.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.3-incubating/Table_Configuration.mdtext
Tue Nov 15 20:53:02 2011
@@ -24,9 +24,8 @@ Notice:    Licensed to the Apache Softwa
 * [Constraints][10]
 * [Bloom Filters][11]
 * [Iterators][12]
-* [Versioning Iterators and Timestamps][13]
-* [Filtering Iterators][14]
-* [Aggregating Iterators][15]
+* [Aggregating Iterators][13]
+* [Block Cache][14]
 
 * * *
 
@@ -157,7 +156,7 @@ Some iterators take additional parameter
 
 Tables support separate Iterator settings to be applied at scan time, upon minor compaction
and upon major compaction. For most uses, tables will have identical iterator settings for
all three to avoid inconsistent results. 
 
-## <a id=Versioning_Iterators_and_Timestamps></a> Versioning Iterators and Timestamps
+### <a id=Versioning_Iterators_and_Timestamps></a> Versioning Iterators and Timestamps
 
 Accumulo provides the capability to manage versioned data through the use of timestamps within
the Key. If a timestamp is not specified in the key created by the client then the system
will set the timestamp to the current time. Two keys with identical rowIDs and columns but
different timestamps are considered two versions of the same key. If two inserts are made
into accumulo with the same rowID, column, and timestamp, then the behavior is non-deterministic.

 
@@ -176,7 +175,7 @@ The version policy can be changed by cha
     table.iterator.majc.vers.opt.maxVersions=3
     
 
-### <a id=Logical_Time></a> Logical Time
+#### <a id=Logical_Time></a> Logical Time
 
 Accumulo 1.2 introduces the concept of logical time. This ensures that timestamps set by
accumulo always move forward. This helps avoid problems caused by TabletServers that have
different time settings. The per tablet counter gives unique one up time stamps on a per mutation
basis. When using time in milliseconds, if two things arrive within the same millisecond then
both receive the same timestamp. 
 
@@ -186,11 +185,11 @@ A table can be configured to use logical
     user@myinstance> createtable -tl logical
     
 
-### <a id=Deletes></a> Deletes
+#### <a id=Deletes></a> Deletes
 
 Deletes are special keys in accumulo that get sorted along will all the other data. When
a delete key is inserted, accumulo will not show anything that has a timestamp less than or
equal to the delete key. During major compaction, any keys older than a delete key are omitted
from the new file created, and the omitted keys are removed from disk as part of the regular
garbage collection process. 
 
-## <a id=Filtering_Iterators></a> Filtering Iterators
+### <a id=Filtering_Iterators></a> Filtering Iterators
 
 When scanning over a set of key-value pairs it is possible to apply an arbitrary filtering
policy through the use of a FilteringIterator. These types of iterators return only key-value
pairs that satisfy the filter logic. Accumulo has two built-in filtering iterators that can
be configured on any table: AgeOff and RegEx. More can be added by writing a Java class that
implements the   
 org.apache.accumulo.core.iterators.filter.Filter interface. 
@@ -306,7 +305,27 @@ Additional Aggregators can be added by c
 **org.apache.accumulo.core.iterators.aggregation.Aggregator** and adding a jar containing
that class to Accumulo's lib directory. 
 
 An example of an aggregator can be found under   
-accumulo/src/examples/main/java/accumulo/examples/aggregation/SortedSetAggregator.java 
+accumulo/src/examples/main/java/org/apache/accumulo/examples/aggregation/SortedSetAggregator.java

+
+## <a id=Block_Cache></a> Block Cache
+
+In order to increase throughput of commonly accessed entries, Accumulo employs a block cache.
This block cache buffers data in memory so that it doesn't have to be read off of disk. The
RFile format that Accumulo prefers is a mix of index blocks and data blocks, where the index
blocks are used to find the appropriate data blocks. Typical queries to Accumulo result in
a binary search over several index blocks followed by a linear scan of one or more data blocks.

+
+The block cache can be configured on a per-table basis, and all tablets hosted on a tablet
server share a single resource pool. To configure the size of the tablet server's block cache,
set the following properties: 
+    
+    
+    tserver.cache.data.size: Specifies the size of the cache for file data blocks.
+    tserver.cache.index.size: Specifies the size of the cache for file indices.
+    
+
+To enable the block cache for your table, set the following properties: 
+    
+    
+    table.cache.block.enable: Determines whether file (data) block cache is enabled.
+    table.cache.index.enable: Determines whether index cache is enabled.
+    
+
+The block cache can have a significant effect on alleviating hot spots, as well as reducing
query latency. It is enabled by default for the !METADATA table. 
 
 * * *
 
@@ -320,7 +339,6 @@ accumulo/src/examples/main/java/accumulo
    [10]: Table_Configuration.html#Constraints
    [11]: Table_Configuration.html#Bloom_Filters
    [12]: Table_Configuration.html#Iterators
-   [13]: Table_Configuration.html#Versioning_Iterators_and_Timestamps
-   [14]: Table_Configuration.html#Filtering_Iterators
-   [15]: Table_Configuration.html#Aggregating_Iterators
+   [13]: Table_Configuration.html#Aggregating_Iterators
+   [14]: Table_Configuration.html#Block_Cache
 

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Administration.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Administration.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Administration.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Administration.mdtext
Tue Nov 15 20:53:02 2011
@@ -136,7 +136,7 @@ The initialization script will prompt yo
 
 Make sure Hadoop is configured on all of the machines in the cluster, including access to
a shared HDFS instance. Make sure HDFS and ZooKeeper are running. Make sure ZooKeeper is configured
and running on at least one machine in the cluster. Start Accumulo using the bin/start-all.sh
script. 
 
-To verify that Accumulo is running, check the Status page as described under _Monitoring_.
In addition, the Shell can provide some information about the status of tables via reading
the !METADATA table. 
+To verify that Accumulo is running, check the Status page as described under *Monitoring*.
In addition, the Shell can provide some information about the status of tables via reading
the !METADATA table. 
 
 ### <a id=Stopping_Accumulo></a> Stopping Accumulo
 

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Contents.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Contents.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Contents.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Contents.mdtext
Tue Nov 15 20:53:02 2011
@@ -78,14 +78,12 @@ Notice:    Licensed to the Apache Softwa
 
         * [Setting Iterators via the Shell][37]
         * [Setting Iterators Programmatically][38]
+        * [Versioning Iterators and Timestamps][39]
+        * [Filters][40]
+        * [Aggregating Iterators][41]
 
-    * [Versioning Iterators and Timestamps][39]
-
-        * [Logical Time][40]
-        * [Deletes][41]
-
-    * [Filters][42]
-    * [Aggregating Iterators][43]
+    * [Block Cache][42]
+    * [Compaction][43]
     * [Pre-splitting tables][44]
     * [Merging tablets][45]
     * [Delete Range][46]
@@ -208,10 +206,10 @@ Notice:    Licensed to the Apache Softwa
    [37]: Table_Configuration.html#Setting_Iterators_via_the_Shell
    [38]: Table_Configuration.html#Setting_Iterators_Programmatically
    [39]: Table_Configuration.html#Versioning_Iterators_and_Timestamps
-   [40]: Table_Configuration.html#Logical_Time
-   [41]: Table_Configuration.html#Deletes
-   [42]: Table_Configuration.html#Filters
-   [43]: Table_Configuration.html#Aggregating_Iterators
+   [40]: Table_Configuration.html#Filters
+   [41]: Table_Configuration.html#Aggregating_Iterators
+   [42]: Table_Configuration.html#Block_Cache
+   [43]: Table_Configuration.html#Compaction
    [44]: Table_Configuration.html#Pre-splitting_tables
    [45]: Table_Configuration.html#Merging_tablets
    [46]: Table_Configuration.html#Delete_Range

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Shell_Commands.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Shell_Commands.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Shell_Commands.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Shell_Commands.mdtext
Tue Nov 15 20:53:02 2011
@@ -395,7 +395,7 @@ Notice:    Licensed to the Apache Softwa
   
     usage: listscans [-?] [-np] [-ts <tablet server>]   
     description: list what scans are currently running in accumulo. See the   
-           accumulo.core.client.admin.ActiveScan javadoc for more information   
+           org.apache.accumulo.core.client.admin.ActiveScan javadoc for more information
  
            about columns.   
       -?,-help  display this help   
       -np,-no-pagination  disables pagination of output   

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Configuration.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Configuration.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Configuration.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Configuration.mdtext
Tue Nov 15 20:53:02 2011
@@ -24,19 +24,18 @@ Notice:    Licensed to the Apache Softwa
 * [Constraints][10]
 * [Bloom Filters][11]
 * [Iterators][12]
-* [Versioning Iterators and Timestamps][13]
-* [Filters][14]
-* [Aggregating Iterators][15]
-* [Pre-splitting tables][16]
-* [Merging tablets][17]
-* [Delete Range][18]
-* [Cloning Tables][19]
+* [Block Cache][13]
+* [Compaction][14]
+* [Pre-splitting tables][15]
+* [Merging tablets][16]
+* [Delete Range][17]
+* [Cloning Tables][18]
 
 * * *
 
 ## <a id=Table_Configuration></a> Table Configuration
 
-Accumulo tables have a few options that can be configured to alter the default behavior of
Accumulo as well as improve performance based on the data stored. These include locality groups,
constraints, and iterators. 
+Accumulo tables have a few options that can be configured to alter the default behavior of
Accumulo as well as improve performance based on the data stored. These include locality groups,
constraints, bloom filters, iterators, and block cache. 
 
 ## <a id=Locality_Groups></a> Locality Groups
 
@@ -161,7 +160,7 @@ Some iterators take additional parameter
 
 Tables support separate Iterator settings to be applied at scan time, upon minor compaction
and upon major compaction. For most uses, tables will have identical iterator settings for
all three to avoid inconsistent results. 
 
-## <a id=Versioning_Iterators_and_Timestamps></a> Versioning Iterators and Timestamps
+### <a id=Versioning_Iterators_and_Timestamps></a> Versioning Iterators and Timestamps
 
 Accumulo provides the capability to manage versioned data through the use of timestamps within
the Key. If a timestamp is not specified in the key created by the client then the system
will set the timestamp to the current time. Two keys with identical rowIDs and columns but
different timestamps are considered two versions of the same key. If two inserts are made
into accumulo with the same rowID, column, and timestamp, then the behavior is non-deterministic.

 
@@ -180,7 +179,7 @@ The version policy can be changed by cha
     table.iterator.majc.vers.opt.maxVersions=3
     
 
-### <a id=Logical_Time></a> Logical Time
+#### <a id=Logical_Time></a> Logical Time
 
 Accumulo 1.2 introduces the concept of logical time. This ensures that timestamps set by
accumulo always move forward. This helps avoid problems caused by TabletServers that have
different time settings. The per tablet counter gives unique one up time stamps on a per mutation
basis. When using time in milliseconds, if two things arrive within the same millisecond then
both receive the same timestamp. When using time in milliseconds, accumulo set times will
still always move forward and never backwards. 
 
@@ -190,11 +189,11 @@ A table can be configured to use logical
     user@myinstance> createtable -tl logical
     
 
-### <a id=Deletes></a> Deletes
+#### <a id=Deletes></a> Deletes
 
 Deletes are special keys in accumulo that get sorted along will all the other data. When
a delete key is inserted, accumulo will not show anything that has a timestamp less than or
equal to the delete key. During major compaction, any keys older than a delete key are omitted
from the new file created, and the omitted keys are removed from disk as part of the regular
garbage collection process. 
 
-## <a id=Filters></a> Filters
+### <a id=Filters></a> Filters
 
 When scanning over a set of key-value pairs it is possible to apply an arbitrary filtering
policy through the use of a Filter. Filters are types of iterators that return only key-value
pairs that satisfy the filter logic. Accumulo has a few built-in filters that can be configured
on any table: AgeOff, ColumnAgeOff, Timestamp, NoVis, and RegEx. More can be added by writing
a Java class that extends the   
 org.apache.accumulo.core.iterators.Filter class. 
@@ -240,7 +239,7 @@ To see the iterator settings for a table
     ---------+------------------------------------------+------------------
     
 
-## <a id=Aggregating_Iterators></a> Aggregating Iterators
+### <a id=Aggregating_Iterators></a> Aggregating Iterators
 
 Accumulo allows aggregating iterators to be configured on tables and column families. When
an aggregating iterator is set, the iterator is applied across the values associated with
any keys that share rowID, column family, and column qualifier. This is similar to the reduce
step in MapReduce, which applied some function to all the values associated with a particular
key. 
 
@@ -287,7 +286,65 @@ Additional Aggregators can be added by c
 **org.apache.accumulo.core.iterators.aggregation.Aggregator** and adding a jar containing
that class to Accumulo's lib directory. 
 
 An example of an aggregator can be found under   
-accumulo/src/examples/main/java/accumulo/examples/aggregation/SortedSetAggregator.java 
+accumulo/src/examples/main/java/org/apache/accumulo/examples/aggregation/SortedSetAggregator.java

+
+## <a id=Block_Cache></a> Block Cache
+
+In order to increase throughput of commonly accessed entries, Accumulo employs a block cache.
This block cache buffers data in memory so that it doesn't have to be read off of disk. The
RFile format that Accumulo prefers is a mix of index blocks and data blocks, where the index
blocks are used to find the appropriate data blocks. Typical queries to Accumulo result in
a binary search over several index blocks followed by a linear scan of one or more data blocks.

+
+The block cache can be configured on a per-table basis, and all tablets hosted on a tablet
server share a single resource pool. To configure the size of the tablet server's block cache,
set the following properties: 
+    
+    
+    tserver.cache.data.size: Specifies the size of the cache for file data blocks.
+    tserver.cache.index.size: Specifies the size of the cache for file indices.
+    
+
+To enable the block cache for your table, set the following properties: 
+    
+    
+    table.cache.block.enable: Determines whether file (data) block cache is enabled.
+    table.cache.index.enable: Determines whether index cache is enabled.
+    
+
+The block cache can have a significant effect on alleviating hot spots, as well as reducing
query latency. It is enabled by default for the !METADATA table. 
+
+## <a id=Compaction></a> Compaction
+
+As data is written to Accumulo it is buffered in memory. The data buffered in memory is eventually
written to HDFS on a per tablet basis. Files can also be added to tablets directly by bulk
import. In the background tablet servers run major compactions to merge multiple files into
one. The tablet server has to decide which tablets to compact and which files within a tablet
to compact. This decision is made using the compaction ratio, which is configurable on a per
table basis. To configure this ratio modify the following property: 
+    
+    
+    table.compaction.major.ratio
+    
+
+Increasing this ratio will result in more files per tablet and less compaction work. More
files per tablet means more higher query latency. So adjusting this ratio is a trade off between
ingest and query performance. The ratio defaults to 3. 
+
+The way the ratio works is that a set of files is compacted into one file if the sum of the
sizes of the files in the set is larger than the ratio multiplied by the size of the largest
file in the set. If this is not true for the set of all files in a tablet, the largest file
is removed from consideration, and the remaining files are considered for compaction. This
is repeated until a compaction is triggered or there are no files left to consider. 
+
+The number of background threads tablet servers use to run major compactions is configurable.
To configure this modify the following property: 
+    
+    
+    tserver.compaction.major.concurrent.max
+    
+
+Also, the number of threads tablet servers use for minor compactions is configurable. To
configure this modify the following property: 
+    
+    
+    tserver.compaction.minor.concurrent.max
+    
+
+The numbers of minor and major compactions running and queued is visible on the Accumulo
monitor page. This allows you to see if compactions are backing up and adjustments to the
above settings are needed. When adjusting the number of threads available for compactions,
consider the number of cores and other tasks running on the nodes such as maps and reduces.

+
+If major compactions are not keeping up, then the number of files per tablet will grow to
a point such that query performance starts to suffer. One way to handle this situation is
to increase the compaction ratio. For example, if the compaction ratio were set to 1, then
every new file added to a tablet by minor compaction would immediately queue the tablet for
major compaction. So if a tablet has a 200M file and minor compaction writes a 1M file, then
the major compaction will attempt to merge the 200M and 1M file. If the tablet server has
lots of tablets trying to do this sort of thing, then major compactions will back up and the
number of files per tablet will start to grow, assuming data is being continuously written.
Increasing the compaction ratio will alleviate backups by lowering the amount of major compaction
work that needs to be done. 
+
+Another option to deal with the files per tablet growing too large is to adjust the following
property: 
+    
+    
+    table.file.max
+    
+
+When a tablet reaches this number of files and needs to flush its in-memory data to disk,
it will choose to do a merging minor compaction. A merging minor compaction will merge the
tablet's smallest file with the data in memory at minor compaction time. Therefore the number
of files will not grow beyond this limit. This will make minor compactions take longer, which
will cause ingest performance to decrease. This can cause ingest to slow down until major
compactions have enough time to catch up. When adjusting this property, also consider adjusting
the compaction ratio. Ideally, merging minor compactions never need to occur and major compactions
will keep up. It is possible to configure the file max and compaction ratio such that only
merging minor compactions occur and major compactions never occur. This should be avoided
because doing only merging minor compactions causes ![$O(N^2)$][19] work to be done. The amount
of work done by major compactions is  ![$O(N*\log_R(N))$][
 20] where *R* is the compaction ratio. 
+
+Compactions can be initiated manually for a table. To initiate a minor compaction, use the
flush command in the shell. To initiate a major compaction, use the compact command in the
shell. The compact command will compact all tablets in a table to one file. Even tablets with
one file are compacted. This is useful for the case where a major compaction filter is configured
for a table. In 1.4 the ability to compact a range of a table was added. To use this feature
specify start and stop rows for the compact command. This will only compact tablets that overlap
the given row range. 
 
 ## <a id=Pre-splitting_tables></a> Pre-splitting tables
 
@@ -422,11 +479,12 @@ The du command in the shell shows how mu
    [10]: Table_Configuration.html#Constraints
    [11]: Table_Configuration.html#Bloom_Filters
    [12]: Table_Configuration.html#Iterators
-   [13]: Table_Configuration.html#Versioning_Iterators_and_Timestamps
-   [14]: Table_Configuration.html#Filters
-   [15]: Table_Configuration.html#Aggregating_Iterators
-   [16]: Table_Configuration.html#Pre-splitting_tables
-   [17]: Table_Configuration.html#Merging_tablets
-   [18]: Table_Configuration.html#Delete_Range
-   [19]: Table_Configuration.html#Cloning_Tables
+   [13]: Table_Configuration.html#Block_Cache
+   [14]: Table_Configuration.html#Compaction
+   [15]: Table_Configuration.html#Pre-splitting_tables
+   [16]: Table_Configuration.html#Merging_tablets
+   [17]: Table_Configuration.html#Delete_Range
+   [18]: Table_Configuration.html#Cloning_Tables
+   [19]: img2.png
+   [20]: img3.png
 

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Design.mdtext
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Design.mdtext?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Design.mdtext
(original)
+++ incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/Table_Design.mdtext
Tue Nov 15 20:53:02 2011
@@ -200,8 +200,8 @@ Of course, fast servers will return thei
    [11]: Table_Design.html#Indexing
    [12]: Table_Design.html#Entity-Attribute_and_Graph_Tables
    [13]: Table_Design.html#Document-Partitioned_Indexing
-   [14]: img2.png
-   [15]: img3.png
-   [16]: img4.png
-   [17]: img5.png
+   [14]: img4.png
+   [15]: img5.png
+   [16]: img6.png
+   [17]: img7.png
 

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img2.png
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img2.png?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
Binary files - no diff available.

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img3.png
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img3.png?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
Binary files - no diff available.

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img4.png
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img4.png?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
Binary files - no diff available.

Modified: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img5.png
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img5.png?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
Binary files - no diff available.

Added: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img6.png
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img6.png?rev=1202418&view=auto
==============================================================================
Binary file - no diff available.

Propchange: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img6.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img7.png
URL: http://svn.apache.org/viewvc/incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img7.png?rev=1202418&view=auto
==============================================================================
Binary file - no diff available.

Propchange: incubator/accumulo/site/trunk/content/accumulo/user_manual_1.4-incubating/img7.png
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Propchange: incubator/accumulo/trunk/docs/src/user_manual/
------------------------------------------------------------------------------
--- svn:ignore (original)
+++ svn:ignore Tue Nov 15 20:53:02 2011
@@ -1 +1,2 @@
 accumulo_user_manual.toc
+accumulo_user_manual.out

Modified: incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex?rev=1202418&r1=1202417&r2=1202418&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex (original)
+++ incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex Tue Nov
15 20:53:02 2011
@@ -456,8 +456,8 @@ never need to occur and major compaction
 configure the file max and compaction ratio such that only merging minor
 compactions occur and major compactions never occur. This should be avoided
 because doing only merging minor compactions causes $O(N^2)$ work to be done.
-The amount of work done by major compactions is $O(N*\log_R(N))$ where $R$ is
-the compaction ratio.
+The amount of work done by major compactions is $O(N*\log_R(N))$ where
+\textit{R} is the compaction ratio.
 
 Compactions can be initiated manually for a table.  To initiate a minor
 compaction, use the flush command in the shell.  To initiate a major compaction,



Mime
View raw message