fluo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [7/8] incubator-fluo-website git commit: Jekyll build from gh-pages:887634d
Date Fri, 28 Oct 2016 16:57:44 GMT
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/f6125fa2/docs/fluo-recipes/1.0.0-incubating/export-queue/index.html
----------------------------------------------------------------------
diff --git a/docs/fluo-recipes/1.0.0-incubating/export-queue/index.html b/docs/fluo-recipes/1.0.0-incubating/export-queue/index.html
new file mode 100644
index 0000000..8bf3399
--- /dev/null
+++ b/docs/fluo-recipes/1.0.0-incubating/export-queue/index.html
@@ -0,0 +1,409 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <link rel="stylesheet" href="/stylesheets/fluo.css">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+    <link rel="canonical" href="https://fluo.apache.org//docs/fluo-recipes/1.0.0-incubating/export-queue/">
+    <link rel="icon" type="image/png" href="/resources/favicon.png">
+    <title>Export Queue Recipe | Apache Fluo</title>
+    <!--[if lt IE 9]>
+      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>
+    <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+      <div class="container">
+        <div class="navbar-header">
+          <div class="navbar-toggle-wrapper visible-xs">
+            <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+            </button>
+          </div>
+          <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+        </div>
+        <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+          <ul class="navbar-nav nav">
+            <li><a href="/release/">Releases</a></li>
+            <li><a href="/tour/">Tour</a></li>
+            <li><a href="/docs/">Docs</a></li>
+            <li><a href="/api/">API</a></li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/getinvolved/">Get Involved</a></li>
+                <li><a href="/news/">News Archive</a></li>
+                <li><a href="/people/">People</a></li>
+                <li><a href="/related-projects/">Related Projects</a></li>
+                <li><a href="/poweredby/">Powered By</a></li>
+              </ul>
+            </li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/how-to-contribute/">How To Contribute</a></li>
+                <li><a href="/release-process/">Release Process</a></li>
+              </ul>
+            </li>
+          </ul>
+          <ul class="navbar-nav nav navbar-right">
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="https://www.apache.org">Apache Homepage</a></li>
+                <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+                <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+                <li><a href="https://www.apache.org/security">Security</a></li>
+                <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+                <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+              </ul>
+            </li>
+          </ul>
+        </div>
+      </div>
+    </div>
+    <div class="container">
+      <div class="row">
+          <div class="col-sm-12">
+            <div class="fluo-doc">
+
+  
+
+  
+
+  <header class="post-header">
+    <h2 class="post-title">Export Queue Recipe</h2>
+  </header>
+
+  <article id="page-content">
+    <h2 id="background">Background</h2>
+
+<p>Fluo is not suited for servicing low latency queries for two reasons.  The
+first reason is that the implementation of transactions are designed for
+throughput.  To get throughput, transactions recover lazily from failures and
+may wait on another transaction that is writing.  Both of these design decisions
+can lead to delays for an individual transaction, but do not negatively impact
+throughput.   The second reason is that Fluo observers executing transactions
+will likely cause a large number of random accesses.  This could lead to high
+response time variability for an individual random access.  This variability
+would not impede throughput but would impede the goal of latency.</p>
+
+<p>One way to make data transformed by Fluo available for low latency queries is
+to export that data to another system.  For example Fluo could be running
+cluster A, continually transforming a large data set, and exporting data to
+Accumulo tables on cluster B.  The tables on cluster B would service user
+queries.  Fluo Recipes has built in support for <a href="/docs/fluo-recipes/1.0.0-incubating/accumulo-export-queue/">exporting to Accumulo</a>,
+however recipe could be used to export to systems other than Accumulo, like
+Redis, Elasticsearch, MySQL, etc.</p>
+
+<p>Exporting data from Fluo is easy to get wrong which is why this recipe exists.
+To understand what can go wrong consider the following example observer
+transaction.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyObserver</span> <span class="kd">extends</span> <span class="n">AbstractObserver</span> <span class="o">{</span>
+
+    <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">TYPEL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">TypeLayer</span><span class="o">(</span><span class="k">new</span> <span class="n">StringEncoder</span><span class="o">());</span>
+
+    <span class="c1">//reperesents a Query system extrnal to Fluo that is updated by Fluo</span>
+    <span class="n">QuerySystem</span> <span class="n">querySystem</span><span class="o">;</span>
+
+    <span class="nd">@Override</span>
+    <span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="n">Bytes</span> <span class="n">row</span><span class="o">,</span> <span class="n">Column</span> <span class="n">col</span><span class="o">)</span> <span class="o">{</span>
+
+        <span class="n">TypedTransactionBase</span> <span class="n">ttx</span> <span class="o">=</span> <span class="n">TYPEL</span><span class="o">.</span><span class="na">wrap</span><span class="o">(</span><span class="n">tx</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">oldCount</span> <span class="o">=</span> <span class="n">ttx</span><span class="o">.</span><span class="na">get</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"counter1"</span><span class="o">).</span><span class="na">toInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">numUpdates</span> <span class="o">=</span> <span class="n">ttx</span><span class="o">.</span><span class="na">get</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"numUpdates"</span><span class="o">).</span><span class="na">toInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">newCount</span> <span class="o">=</span> <span class="n">oldCount</span> <span class="o">+</span> <span class="n">numUpdates</span><span class="o">;</span>
+        <span class="n">ttx</span><span class="o">.</span><span class="na">mutate</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"counter1"</span><span class="o">).</span><span class="na">set</span><span class="o">(</span><span class="n">newCount</span><span class="o">);</span>
+        <span class="n">ttx</span><span class="o">.</span><span class="na">mutate</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"numUpdates"</span><span class="o">).</span><span class="na">set</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+
+        <span class="c1">//Build an inverted index in the query system, based on count from the</span>
+        <span class="c1">//meta:counter1 column in fluo.  Do this by creating rows for the</span>
+        <span class="c1">//external query system based on the count.        </span>
+        <span class="n">String</span> <span class="n">oldCountRow</span> <span class="o">=</span> <span class="n">String</span><span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"%06d"</span><span class="o">,</span> <span class="n">oldCount</span><span class="o">);</span>
+        <span class="n">String</span> <span class="n">newCountRow</span> <span class="o">=</span> <span class="n">String</span><span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"%06d"</span><span class="o">,</span> <span class="n">newCount</span><span class="o">);</span>
+        
+        <span class="c1">//add a new entry to the inverted index</span>
+        <span class="n">querySystem</span><span class="o">.</span><span class="na">insertRow</span><span class="o">(</span><span class="n">newCountRow</span><span class="o">,</span> <span class="n">row</span><span class="o">);</span>
+        <span class="c1">//remove the old entry from the inverted index</span>
+        <span class="n">querySystem</span><span class="o">.</span><span class="na">deleteRow</span><span class="o">(</span><span class="n">oldCountRow</span><span class="o">,</span> <span class="n">row</span><span class="o">);</span>
+    <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>The above example would keep the external index up to date beautifully as long
+as the following conditions are met.</p>
+
+<ul>
+  <li>Threads executing transactions always complete successfully.</li>
+  <li>Only a single thread ever responds to a notification.</li>
+</ul>
+
+<p>However these conditions are not guaranteed by Fluo.  Multiple threads may
+attempt to process a notification concurrently (only one may succeed).  Also at
+any point in time a transaction may fail (for example the computer executing it
+may reboot).   Both of these problems will occur and will lead to corruption of
+the external index in the example.  The inverted index and Fluo  will become
+inconsistent.  The inverted index will end up with multiple entries (that are
+never cleaned up) for single entity even though the intent is to only have one.</p>
+
+<p>The root of the problem in the example above is that its exporting uncommitted
+data.  There is no guarantee that setting the column <code class="highlighter-rouge">&lt;row&gt;:meta:counter1</code> to
+<code class="highlighter-rouge">newCount</code> will succeed until the transaction is successfully committed.
+However, <code class="highlighter-rouge">newCountRow</code> is derived from <code class="highlighter-rouge">newCount</code> and written to the external query
+system before the transaction is committed (Note : for observers, the
+transaction is committed by the framework after <code class="highlighter-rouge">process(...)</code> is called).  So
+if the transaction fails, the next time it runs it could compute a completely
+different value for <code class="highlighter-rouge">newCountRow</code> (and it would not delete what was written by the
+failed transaction).</p>
+
+<h2 id="solution">Solution</h2>
+
+<p>The simple solution to the problem of exporting uncommitted data is to only
+export committed data.  There are multiple ways to accomplish this.  This
+recipe offers a reusable implementation of one method.  This recipe has the
+following elements:</p>
+
+<ul>
+  <li>An export queue that transactions can add key/values to.  Only if the transaction commits successfully will the key/value end up in the queue.  A Fluo application can have multiple export queues, each one must have a unique id.</li>
+  <li>When a key/value is added to the export queue, its given a sequence number.  This sequence number is based on the transactions start timestamp.</li>
+  <li>Each export queue is configured with an observer that processes key/values that were successfully committed to the queue.</li>
+  <li>When key/values in an export queue are processed, they are deleted so the export queue does not keep any long term data.</li>
+  <li>Key/values in an export queue are placed in buckets.  This is done so that all of the updates in a bucket can be processed in a single transaction.  This allows an efficient implementation of this recipe in Fluo.  It can also lead to efficiency in a system being exported to, if the system can benefit from batching updates.  The number of buckets in an export queue is configurable.</li>
+</ul>
+
+<p>There are three requirements for using this recipe :</p>
+
+<ul>
+  <li>Must configure export queues before initializing a Fluo application.</li>
+  <li>Transactions adding to an export queue must get an instance of the queue using its unique QID.</li>
+  <li>Must implement a class that extends <a href="https://static.javadoc.io/org.apache.fluo/fluo-recipes-core/1.0.0-incubating/org/apache/fluo/recipes/core/export/Exporter.html">Exporter</a> in order to process exports.</li>
+</ul>
+
+<h2 id="schema">Schema</h2>
+
+<p>Each export queue stores its data in the Fluo table in a contiguous row range.
+This row range is defined by using the export queue id as a row prefix for all
+data in the export queue.  So the row range defined by the export queue id
+should not be used by anything else.</p>
+
+<p>All data stored in an export queue is <a href="transient.md">transient</a>. When an export
+queue is configured, it will recommend split points using the <a href="table-optimization.md">table
+optimization process</a>.  The number of splits generated
+by this process can be controlled by setting the number of buckets per tablet
+when configuring an export queue.</p>
+
+<h2 id="example-use">Example Use</h2>
+
+<p>This example will show how to build an inverted index in an external
+query system using an export queue.  The class below is simple POJO to hold the
+count update, this will be used as the value for the export queue.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kd">class</span> <span class="nc">CountUpdate</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="kt">int</span> <span class="n">oldCount</span><span class="o">;</span>
+  <span class="kd">public</span> <span class="kt">int</span> <span class="n">newCount</span><span class="o">;</span>
+  
+  <span class="kd">public</span> <span class="nf">CountUpdate</span><span class="o">(</span><span class="kt">int</span> <span class="n">oc</span><span class="o">,</span> <span class="kt">int</span> <span class="n">nc</span><span class="o">)</span> <span class="o">{</span>
+    <span class="k">this</span><span class="o">.</span><span class="na">oldCount</span> <span class="o">=</span> <span class="n">oc</span><span class="o">;</span>
+    <span class="k">this</span><span class="o">.</span><span class="na">newCount</span> <span class="o">=</span> <span class="n">nc</span><span class="o">;</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>The following code shows how to configure an export queue.  This code will
+modify the FluoConfiguration object with options needed for the export queue.
+This FluoConfiguration object should be used to initialize the fluo
+application.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code>   <span class="n">FluoConfiguration</span> <span class="n">fluoConfig</span> <span class="o">=</span> <span class="o">...;</span>
+
+   <span class="c1">//queue id "ici" means inverted count index, would probably use static final in real application</span>
+   <span class="n">String</span> <span class="n">exportQueueID</span> <span class="o">=</span> <span class="s">"ici"</span><span class="o">;</span>  
+   <span class="n">Class</span><span class="o">&lt;</span><span class="n">CountExporter</span><span class="o">&gt;</span> <span class="n">exporterType</span> <span class="o">=</span> <span class="n">CountExporter</span><span class="o">.</span><span class="na">class</span><span class="o">;</span>
+   <span class="n">Class</span><span class="o">&lt;</span><span class="n">Bytes</span><span class="o">&gt;</span> <span class="n">keyType</span> <span class="o">=</span> <span class="n">Bytes</span><span class="o">.</span><span class="na">class</span><span class="o">;</span>
+   <span class="n">Class</span><span class="o">&lt;</span><span class="n">CountUpdate</span><span class="o">&gt;</span> <span class="n">valueType</span> <span class="o">=</span> <span class="n">CountUpdate</span><span class="o">.</span><span class="na">class</span><span class="o">;</span>
+   <span class="kt">int</span> <span class="n">numBuckets</span> <span class="o">=</span> <span class="mi">1009</span><span class="o">;</span>
+   <span class="c1">//the desired number of tablets to create when applying table optimizations</span>
+   <span class="kt">int</span> <span class="n">numTablets</span> <span class="o">=</span> <span class="mi">100</span><span class="o">;</span>
+
+   <span class="n">ExportQueue</span><span class="o">.</span><span class="na">Options</span> <span class="n">eqOptions</span> <span class="o">=</span>
+        <span class="k">new</span> <span class="n">ExportQueue</span><span class="o">.</span><span class="na">Options</span><span class="o">(</span><span class="n">exportQueueId</span><span class="o">,</span> <span class="n">exporterType</span><span class="o">,</span> <span class="n">keyType</span><span class="o">,</span> <span class="n">valueType</span><span class="o">,</span> <span class="n">numBuckets</span><span class="o">)</span>
+          <span class="o">.</span><span class="na">setsetBucketsPerTablet</span><span class="o">(</span><span class="n">numBuckets</span><span class="o">/</span><span class="n">numTablets</span><span class="o">);</span>
+   <span class="n">ExportQueue</span><span class="o">.</span><span class="na">configure</span><span class="o">(</span><span class="n">fluoConfig</span><span class="o">,</span> <span class="n">eqOptions</span><span class="o">);</span>
+
+   <span class="c1">//initialize Fluo using fluoConfig</span>
+</code></pre>
+</div>
+
+<p>Below is updated version of the observer from above thats now using the export
+queue.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyObserver</span> <span class="kd">extends</span> <span class="n">AbstractObserver</span> <span class="o">{</span>
+
+    <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">TYPEL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">TypeLayer</span><span class="o">(</span><span class="k">new</span> <span class="n">StringEncoder</span><span class="o">());</span>
+
+    <span class="kd">private</span> <span class="n">ExportQueue</span><span class="o">&lt;</span><span class="n">Bytes</span><span class="o">,</span> <span class="n">CountUpdate</span><span class="o">&gt;</span> <span class="n">exportQueue</span><span class="o">;</span>
+
+    <span class="nd">@Override</span>
+    <span class="kd">public</span> <span class="kt">void</span> <span class="nf">init</span><span class="o">(</span><span class="n">Context</span> <span class="n">context</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
+      <span class="n">exportQueue</span> <span class="o">=</span> <span class="n">ExportQueue</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="s">"ici"</span><span class="o">,</span> <span class="n">context</span><span class="o">.</span><span class="na">getAppConfiguration</span><span class="o">());</span>
+    <span class="o">}</span>
+
+    <span class="nd">@Override</span>
+    <span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="n">Bytes</span> <span class="n">row</span><span class="o">,</span> <span class="n">Column</span> <span class="n">col</span><span class="o">)</span> <span class="o">{</span>
+
+        <span class="n">TypedTransactionBase</span> <span class="n">ttx</span> <span class="o">=</span> <span class="n">TYPEL</span><span class="o">.</span><span class="na">wrap</span><span class="o">(</span><span class="n">tx</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">oldCount</span> <span class="o">=</span> <span class="n">ttx</span><span class="o">.</span><span class="na">get</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"counter1"</span><span class="o">).</span><span class="na">toInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">numUpdates</span> <span class="o">=</span> <span class="n">ttx</span><span class="o">.</span><span class="na">get</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"numUpdates"</span><span class="o">).</span><span class="na">toInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">newCount</span> <span class="o">=</span> <span class="n">oldCount</span> <span class="o">+</span> <span class="n">numUpdates</span><span class="o">;</span>
+        <span class="n">ttx</span><span class="o">.</span><span class="na">mutate</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"counter1"</span><span class="o">).</span><span class="na">set</span><span class="o">(</span><span class="n">newCount</span><span class="o">);</span>
+        <span class="n">ttx</span><span class="o">.</span><span class="na">mutate</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"numUpdates"</span><span class="o">).</span><span class="na">set</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+
+        <span class="c1">//Because the update to the export queue is part of the transaction,</span>
+        <span class="c1">//either the update to meta:counter1 is made and an entry is added to</span>
+        <span class="c1">//the export queue or neither happens.</span>
+        <span class="n">exportQueue</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">tx</span><span class="o">,</span> <span class="n">row</span><span class="o">,</span> <span class="k">new</span> <span class="n">CountUpdate</span><span class="o">(</span><span class="n">oldCount</span><span class="o">,</span> <span class="n">newCount</span><span class="o">));</span>
+    <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>The observer setup for the export queue will call the <code class="highlighter-rouge">processExports()</code> method
+on the class below to process entries in the export queue.  It possible the
+call to <code class="highlighter-rouge">processExports()</code> can fail part way through and/or be called multiple
+times.  In the case of failures the exporter will eventually be called again
+with the exact same data.  The possibility of the same export entry being
+processed on multiple computer at different times can cause exports to arrive
+out of order.  The system receiving exports has to be resilient to data
+arriving out of order and multiple times.  The purpose of the sequence number
+is to help systems receiving data from Fluo process out of order and redundant
+data.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code>  <span class="kd">public</span> <span class="kd">class</span> <span class="nc">CountExporter</span> <span class="kd">extends</span> <span class="n">Exporter</span><span class="o">&lt;</span><span class="n">Bytes</span><span class="o">,</span> <span class="n">CountUpdate</span><span class="o">&gt;</span> <span class="o">{</span>
+    <span class="c1">//represents the external query system we want to update from Fluo</span>
+    <span class="n">QuerySystem</span> <span class="n">querySystem</span><span class="o">;</span>
+
+    <span class="nd">@Override</span>
+    <span class="kd">protected</span> <span class="kt">void</span> <span class="nf">processExports</span><span class="o">(</span><span class="n">Iterator</span><span class="o">&lt;</span><span class="n">SequencedExport</span><span class="o">&lt;</span><span class="n">Bytes</span><span class="o">,</span> <span class="n">CountUpdate</span><span class="o">&gt;&gt;</span> <span class="n">exportIterator</span><span class="o">)</span> <span class="o">{</span>
+      <span class="n">BatchUpdater</span> <span class="n">batchUpdater</span> <span class="o">=</span> <span class="n">querySystem</span><span class="o">.</span><span class="na">getBatchUpdater</span><span class="o">();</span>
+      <span class="k">while</span><span class="o">(</span><span class="n">exportIterator</span><span class="o">.</span><span class="na">hasNext</span><span class="o">()){</span>
+        <span class="n">SequencedExport</span><span class="o">&lt;</span><span class="n">Bytes</span><span class="o">,</span> <span class="n">CountUpdate</span><span class="o">&gt;</span> <span class="n">exportEntry</span> <span class="o">=</span>  <span class="n">exportItertor</span><span class="o">.</span><span class="na">next</span><span class="o">();</span>
+        <span class="n">Bytes</span> <span class="n">row</span> <span class="o">=</span>  <span class="n">exportEntry</span><span class="o">.</span><span class="na">getKey</span><span class="o">();</span>
+        <span class="n">UpdateCount</span> <span class="n">uc</span> <span class="o">=</span> <span class="n">exportEntry</span><span class="o">.</span><span class="na">getValue</span><span class="o">();</span>
+        <span class="kt">long</span> <span class="n">seqNum</span> <span class="o">=</span> <span class="n">exportEntry</span><span class="o">.</span><span class="na">getSequence</span><span class="o">();</span>
+
+        <span class="n">String</span> <span class="n">oldCountRow</span> <span class="o">=</span> <span class="n">String</span><span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"%06d"</span><span class="o">,</span> <span class="n">uc</span><span class="o">.</span><span class="na">oldCount</span><span class="o">);</span>
+        <span class="n">String</span> <span class="n">newCountRow</span> <span class="o">=</span> <span class="n">String</span><span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"%06d"</span><span class="o">,</span> <span class="n">uc</span><span class="o">.</span><span class="na">newCount</span><span class="o">);</span>
+
+        <span class="c1">//add a new entry to the inverted index</span>
+        <span class="n">batchUpdater</span><span class="o">.</span><span class="na">insertRow</span><span class="o">(</span><span class="n">newCountRow</span><span class="o">,</span> <span class="n">row</span><span class="o">,</span> <span class="n">seqNum</span><span class="o">);</span>
+        <span class="c1">//remove the old entry from the inverted index</span>
+        <span class="n">batchUpdater</span><span class="o">.</span><span class="na">deleteRow</span><span class="o">(</span><span class="n">oldCountRow</span><span class="o">,</span> <span class="n">row</span><span class="o">,</span> <span class="n">seqNum</span><span class="o">);</span>
+      <span class="o">}</span>
+
+      <span class="c1">//flush all of the updates to the external query system</span>
+      <span class="n">batchUpdater</span><span class="o">.</span><span class="na">close</span><span class="o">();</span>
+    <span class="o">}</span>
+  <span class="o">}</span>
+</code></pre>
+</div>
+
+<h2 id="concurrency">Concurrency</h2>
+
+<p>Additions to the export queue will never collide.  If two transactions add the
+same key at around the same time and successfully commit, then two entries with
+different sequence numbers will always be added to the queue.  The sequence
+number is based on the start timestamp of the transactions.</p>
+
+<p>If the key used to add items to the export queue is deterministically derived
+from something the transaction is writing to, then that will cause a collision.
+For example consider the following interleaving of two transactions adding to
+the same export queue in a manner that will collide. Note, TH1 is shorthand for
+thread 1, ek() is a function the creates the export key, and ev() is a function
+that creates the export value.</p>
+
+<ol>
+  <li>TH1 : key1 = ek(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>)</li>
+  <li>TH1 : val1 = ev(tx1.get(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>), tx1.get(<code class="highlighter-rouge">rowA</code>,<code class="highlighter-rouge">fam1:qual2</code>))</li>
+  <li>TH1 : exportQueueA.add(tx1, key1, val1)</li>
+  <li>TH2 : key2 = ek(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>)</li>
+  <li>TH2 : val2 = ev(tx2.get(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>), tx2.get(<code class="highlighter-rouge">rowB</code>,<code class="highlighter-rouge">fam1:qual2</code>))</li>
+  <li>TH2 : exportQueueA.add(tx2, key2, val2)</li>
+  <li>TH1 : tx1.set(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>, val1)</li>
+  <li>TH2 : tx2.set(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>, val2)</li>
+</ol>
+
+<p>In the example above only one transaction will succeed because both are setting
+<code class="highlighter-rouge">row1 fam1:qual1</code>.  Since adding to the export queue is part of the
+transaction, only the transaction that succeeds will add something to the
+queue.  If the funtion ek() in the example is deterministic, then both
+transactions would have been trying to add the same key to the export queue.</p>
+
+<p>With the above method, we know that transactions adding entries to the queue for
+the same key must have executed <a href="https://en.wikipedia.org/wiki/Serializability">serially</a>. Knowing that transactions which
+added the same key did not overlap in time makes reasoning about those export
+entries very simple.</p>
+
+<p>The example below is a slight modification of the example above.  In this
+example both transactions will successfully add entries to the queue using the
+same key.  Both transactions succeed because they are writing to different
+cells (<code class="highlighter-rouge">rowB fam1:qual2</code> and <code class="highlighter-rouge">rowA fam1:qual2</code>).  This approach makes it more
+difficult to reason about export entries with the same key, because the
+transactions adding those entries could have overlapped in time.  This is an
+example of write skew mentioned in the Percolater paper.</p>
+
+<ol>
+  <li>TH1 : key1 = ek(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>)</li>
+  <li>TH1 : val1 = ev(tx1.get(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>), tx1.get(<code class="highlighter-rouge">rowA</code>,<code class="highlighter-rouge">fam1:qual2</code>))</li>
+  <li>TH1 : exportQueueA.add(tx1, key1, val1)</li>
+  <li>TH2 : key2 = ek(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>)</li>
+  <li>TH2 : val2 = ev(tx2.get(<code class="highlighter-rouge">row1</code>,<code class="highlighter-rouge">fam1:qual1</code>), tx2.get(<code class="highlighter-rouge">rowB</code>,<code class="highlighter-rouge">fam1:qual2</code>))</li>
+  <li>TH2 : exportQueueA.add(tx2, key2, val2)</li>
+  <li>TH1 : tx1.set(<code class="highlighter-rouge">rowA</code>,<code class="highlighter-rouge">fam1:qual2</code>, val1)</li>
+  <li>TH2 : tx2.set(<code class="highlighter-rouge">rowB</code>,<code class="highlighter-rouge">fam1:qual2</code>, val2)</li>
+</ol>
+
+
+  </article>
+
+</div>
+
+          </div>
+      </div>
+      <hr>
+      <div class="row footer">
+        <div class="col-sm-12 text-center">
+          <div class="center-block">
+          <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+          Copyright &copy; 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache&nbsp;License,&nbsp;Version&nbsp;2.0</a>
+          </div>
+        </div>
+      </div>
+    </div>
+    <script src="/javascripts/jquery.min.js"></script>
+    <script src="/javascripts/bootstrap.min.js"></script>
+    <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+  ga('create', 'UA-55360307-1', 'auto');
+  ga('send', 'pageview');
+
+</script>
+
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/f6125fa2/docs/fluo-recipes/1.0.0-incubating/index.html
----------------------------------------------------------------------
diff --git a/docs/fluo-recipes/1.0.0-incubating/index.html b/docs/fluo-recipes/1.0.0-incubating/index.html
new file mode 100644
index 0000000..dbed369
--- /dev/null
+++ b/docs/fluo-recipes/1.0.0-incubating/index.html
@@ -0,0 +1,202 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <link rel="stylesheet" href="/stylesheets/fluo.css">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+    <link rel="canonical" href="https://fluo.apache.org//docs/fluo-recipes/1.0.0-incubating/">
+    <link rel="icon" type="image/png" href="/resources/favicon.png">
+    <title>Fluo Recipes 1.0.0-incubating Documentation | Apache Fluo</title>
+    <!--[if lt IE 9]>
+      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>
+    <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+      <div class="container">
+        <div class="navbar-header">
+          <div class="navbar-toggle-wrapper visible-xs">
+            <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+            </button>
+          </div>
+          <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+        </div>
+        <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+          <ul class="navbar-nav nav">
+            <li><a href="/release/">Releases</a></li>
+            <li><a href="/tour/">Tour</a></li>
+            <li><a href="/docs/">Docs</a></li>
+            <li><a href="/api/">API</a></li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/getinvolved/">Get Involved</a></li>
+                <li><a href="/news/">News Archive</a></li>
+                <li><a href="/people/">People</a></li>
+                <li><a href="/related-projects/">Related Projects</a></li>
+                <li><a href="/poweredby/">Powered By</a></li>
+              </ul>
+            </li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/how-to-contribute/">How To Contribute</a></li>
+                <li><a href="/release-process/">Release Process</a></li>
+              </ul>
+            </li>
+          </ul>
+          <ul class="navbar-nav nav navbar-right">
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="https://www.apache.org">Apache Homepage</a></li>
+                <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+                <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+                <li><a href="https://www.apache.org/security">Security</a></li>
+                <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+                <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+              </ul>
+            </li>
+          </ul>
+        </div>
+      </div>
+    </div>
+    <div class="container">
+      <div class="row">
+          <div class="col-sm-12">
+            <div class="fluo-doc">
+
+  
+
+  
+
+  <header class="post-header">
+    <h2 class="post-title">Fluo Recipes 1.0.0-incubating Documentation</h2>
+  </header>
+
+  <article id="page-content">
+    <p><strong>Fluo Recipes are common code for <a href="https://fluo.apache.org/">Apache Fluo</a> application developers.</strong></p>
+
+<p>Fluo Recipes build on the <a href="/api/">Fluo API</a> to offer additinal functionality to
+developers. They are published seperately from Fluo on their own release schedule.
+This allows Fluo Recipes to iterate and innovate faster than Fluo (which will maintain
+a more minimal API on a slower release cycle).</p>
+
+<h3 id="documentation">Documentation</h3>
+
+<p>Recipes are documented below and in the <a href="/api/">Recipes API docs</a>.</p>
+
+<ul>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/cfm/">Collision Free Map</a> - A recipe for making many to many updates.</li>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/export-queue/">Export Queue</a> - A recipe for exporting data from Fluo to external systems.</li>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/row-hasher/">Row Hash Prefix</a> - A recipe for spreading data evenly in a row prefix.</li>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/recording-tx/">RecordingTransaction</a> - A wrapper for a Fluo transaction that records all transaction
+operations to a log which can be used to export data from Fluo.</li>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/testing/">Testing</a> Some code to help write Fluo Integration test.</li>
+</ul>
+
+<p>Recipes have common needs that are broken down into the following reusable components.</p>
+
+<ul>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/serialization/">Serialization</a> - Common code for serializing POJOs.</li>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/transient/">Transient Ranges</a> - Standardized process for dealing with transient data.</li>
+  <li><a href="/docs/fluo-recipes/1.0.0-incubating/table-optimization/">Table optimization</a> - Standardized process for optimizing the Fluo table.</li>
+</ul>
+
+<h3 id="usage">Usage</h3>
+
+<p>The Fluo Recipes project publishes multiple jars to Maven Central for each release.
+The <code class="highlighter-rouge">fluo-recipes-core</code> jar is the primary jar. It is where most recipes live and where
+they are placed by default if they have minimal dependencies beyond the Fluo API.</p>
+
+<p>Fluo Recipes with dependencies that bring in many transitive dependencies publish
+their own jar. For example, recipes that depend on Apache Spark are published in the
+<code class="highlighter-rouge">fluo-recipes-spark</code> jar.  If you don’t plan on using code in the <code class="highlighter-rouge">fluo-recipes-spark</code>
+jar, you should avoid including it in your pom.xml to avoid a transitive dependency on
+Spark.</p>
+
+<p>Below is a sample Maven POM containing all possible Fluo Recipes dependencies:</p>
+
+<div class="language-xml highlighter-rouge"><pre class="highlight"><code>  <span class="nt">&lt;properties&gt;</span>
+    <span class="nt">&lt;fluo-recipes.version&gt;</span>1.0.0-incubating<span class="nt">&lt;/fluo-recipes.version&gt;</span>
+  <span class="nt">&lt;/properties&gt;</span>
+
+  <span class="nt">&lt;dependencies&gt;</span>
+    <span class="c">&lt;!-- Required. Contains recipes that are only depend on the Fluo API --&gt;</span>
+    <span class="nt">&lt;dependency&gt;</span>
+      <span class="nt">&lt;groupId&gt;</span>org.apache.fluo<span class="nt">&lt;/groupId&gt;</span>
+      <span class="nt">&lt;artifactId&gt;</span>fluo-recipes-core<span class="nt">&lt;/artifactId&gt;</span>
+      <span class="nt">&lt;version&gt;</span>${fluo-recipes.version}<span class="nt">&lt;/version&gt;</span>
+    <span class="nt">&lt;/dependency&gt;</span>
+    <span class="c">&lt;!-- Optional. Serialization code that depends on Kryo --&gt;</span>
+    <span class="nt">&lt;dependency&gt;</span>
+      <span class="nt">&lt;groupId&gt;</span>org.apache.fluo<span class="nt">&lt;/groupId&gt;</span>
+      <span class="nt">&lt;artifactId&gt;</span>fluo-recipes-kryo<span class="nt">&lt;/artifactId&gt;</span>
+      <span class="nt">&lt;version&gt;</span>${fluo-recipes.version}<span class="nt">&lt;/version&gt;</span>
+    <span class="nt">&lt;/dependency&gt;</span>
+    <span class="c">&lt;!-- Optional. Common code for using Fluo with Accumulo --&gt;</span>
+    <span class="nt">&lt;dependency&gt;</span>
+      <span class="nt">&lt;groupId&gt;</span>org.apache.fluo<span class="nt">&lt;/groupId&gt;</span>
+      <span class="nt">&lt;artifactId&gt;</span>fluo-recipes-accumulo<span class="nt">&lt;/artifactId&gt;</span>
+      <span class="nt">&lt;version&gt;</span>${fluo-recipes.version}<span class="nt">&lt;/version&gt;</span>
+    <span class="nt">&lt;/dependency&gt;</span>
+    <span class="c">&lt;!-- Optional. Common code for using Fluo with Spark --&gt;</span>
+    <span class="nt">&lt;dependency&gt;</span>
+      <span class="nt">&lt;groupId&gt;</span>org.apache.fluo<span class="nt">&lt;/groupId&gt;</span>
+      <span class="nt">&lt;artifactId&gt;</span>fluo-recipes-spark<span class="nt">&lt;/artifactId&gt;</span>
+      <span class="nt">&lt;version&gt;</span>${fluo-recipes.version}<span class="nt">&lt;/version&gt;</span>
+    <span class="nt">&lt;/dependency&gt;</span>
+    <span class="c">&lt;!-- Optional. Common code for writing Fluo integration tests --&gt;</span>
+    <span class="nt">&lt;dependency&gt;</span>
+      <span class="nt">&lt;groupId&gt;</span>org.apache.fluo<span class="nt">&lt;/groupId&gt;</span>
+      <span class="nt">&lt;artifactId&gt;</span>fluo-recipes-test<span class="nt">&lt;/artifactId&gt;</span>
+      <span class="nt">&lt;version&gt;</span>${fluo-recipes.version}<span class="nt">&lt;/version&gt;</span>
+      <span class="nt">&lt;scope&gt;</span>test<span class="nt">&lt;/scope&gt;</span>
+    <span class="nt">&lt;/dependency&gt;</span>
+  <span class="nt">&lt;/dependencies&gt;</span>
+</code></pre>
+</div>
+
+
+  </article>
+
+</div>
+
+          </div>
+      </div>
+      <hr>
+      <div class="row footer">
+        <div class="col-sm-12 text-center">
+          <div class="center-block">
+          <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+          Copyright &copy; 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache&nbsp;License,&nbsp;Version&nbsp;2.0</a>
+          </div>
+        </div>
+      </div>
+    </div>
+    <script src="/javascripts/jquery.min.js"></script>
+    <script src="/javascripts/bootstrap.min.js"></script>
+    <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+  ga('create', 'UA-55360307-1', 'auto');
+  ga('send', 'pageview');
+
+</script>
+
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/f6125fa2/docs/fluo-recipes/1.0.0-incubating/recording-tx/index.html
----------------------------------------------------------------------
diff --git a/docs/fluo-recipes/1.0.0-incubating/recording-tx/index.html b/docs/fluo-recipes/1.0.0-incubating/recording-tx/index.html
new file mode 100644
index 0000000..1580e2c
--- /dev/null
+++ b/docs/fluo-recipes/1.0.0-incubating/recording-tx/index.html
@@ -0,0 +1,187 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <link rel="stylesheet" href="/stylesheets/fluo.css">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+    <link rel="canonical" href="https://fluo.apache.org//docs/fluo-recipes/1.0.0-incubating/recording-tx/">
+    <link rel="icon" type="image/png" href="/resources/favicon.png">
+    <title>RecordingTransaction recipe | Apache Fluo</title>
+    <!--[if lt IE 9]>
+      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>
+    <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+      <div class="container">
+        <div class="navbar-header">
+          <div class="navbar-toggle-wrapper visible-xs">
+            <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+            </button>
+          </div>
+          <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+        </div>
+        <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+          <ul class="navbar-nav nav">
+            <li><a href="/release/">Releases</a></li>
+            <li><a href="/tour/">Tour</a></li>
+            <li><a href="/docs/">Docs</a></li>
+            <li><a href="/api/">API</a></li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/getinvolved/">Get Involved</a></li>
+                <li><a href="/news/">News Archive</a></li>
+                <li><a href="/people/">People</a></li>
+                <li><a href="/related-projects/">Related Projects</a></li>
+                <li><a href="/poweredby/">Powered By</a></li>
+              </ul>
+            </li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/how-to-contribute/">How To Contribute</a></li>
+                <li><a href="/release-process/">Release Process</a></li>
+              </ul>
+            </li>
+          </ul>
+          <ul class="navbar-nav nav navbar-right">
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="https://www.apache.org">Apache Homepage</a></li>
+                <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+                <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+                <li><a href="https://www.apache.org/security">Security</a></li>
+                <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+                <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+              </ul>
+            </li>
+          </ul>
+        </div>
+      </div>
+    </div>
+    <div class="container">
+      <div class="row">
+          <div class="col-sm-12">
+            <div class="fluo-doc">
+
+  
+
+  
+
+  <header class="post-header">
+    <h2 class="post-title">RecordingTransaction recipe</h2>
+  </header>
+
+  <article id="page-content">
+    <p>A <code class="highlighter-rouge">RecordingTransaction</code> is an implementation of <code class="highlighter-rouge">Transaction</code> that logs all transaction operations
+(i.e GET, SET, or DELETE) to a <code class="highlighter-rouge">TxLog</code> object for later uses such as exporting data.  The code below
+shows how a RecordingTransaction is created by wrapping a Transaction object:</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">RecordingTransactionBase</span> <span class="n">rtx</span> <span class="o">=</span> <span class="n">RecordingTransactionBase</span><span class="o">.</span><span class="na">wrap</span><span class="o">(</span><span class="n">tx</span><span class="o">);</span>
+</code></pre>
+</div>
+
+<p>A predicate function can be passed to wrap method to select which log entries to record.  The code
+below only records log entries whose column family is <code class="highlighter-rouge">meta</code>:</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">RecordingTransactionBase</span> <span class="n">rtx</span> <span class="o">=</span> <span class="n">RecordingTransactionBase</span><span class="o">.</span><span class="na">wrap</span><span class="o">(</span><span class="n">tx</span><span class="o">,</span>
+                               <span class="n">le</span> <span class="o">-&gt;</span> <span class="n">le</span><span class="o">.</span><span class="na">getColumn</span><span class="o">().</span><span class="na">getFamily</span><span class="o">().</span><span class="na">toString</span><span class="o">().</span><span class="na">equals</span><span class="o">(</span><span class="s">"meta"</span><span class="o">));</span>
+</code></pre>
+</div>
+
+<p>After creating a RecordingTransaction, users can use it as they would use a Transaction object.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">Bytes</span> <span class="n">value</span> <span class="o">=</span> <span class="n">rtx</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">Bytes</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="s">"r1"</span><span class="o">),</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"cf1"</span><span class="o">,</span> <span class="s">"cq1"</span><span class="o">));</span>
+</code></pre>
+</div>
+
+<p>While SET or DELETE operations are always recorded to the log, GET operations are only recorded if a
+value was found at the requested row/column.  Also, if a GET method returns an iterator, only the GET
+operations that are retrieved from the iterator are logged.  GET operations are logged as they are
+necessary if you want to determine the changes made by the transaction.</p>
+
+<p>When you are done operating on the transaction, you can retrieve the TxLog using the following code:</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="n">TxLog</span> <span class="n">myTxLog</span> <span class="o">=</span> <span class="n">rtx</span><span class="o">.</span><span class="na">getTxLog</span><span class="o">()</span>
+</code></pre>
+</div>
+
+<p>Below is example code of how a RecordingTransaction can be used in an observer to record all operations
+performed by the transaction in a TxLog.  In this example, a GET (if data exists) and SET operation
+will be logged.  This TxLog can be added to an export queue and later used to export updates from 
+Fluo.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyObserver</span> <span class="kd">extends</span> <span class="n">AbstractObserver</span> <span class="o">{</span>
+
+    <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">TYPEL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">TypeLayer</span><span class="o">(</span><span class="k">new</span> <span class="n">StringEncoder</span><span class="o">());</span>
+    
+    <span class="kd">private</span> <span class="n">ExportQueue</span><span class="o">&lt;</span><span class="n">Bytes</span><span class="o">,</span> <span class="n">TxLog</span><span class="o">&gt;</span> <span class="n">exportQueue</span><span class="o">;</span>
+
+    <span class="nd">@Override</span>
+    <span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="n">Bytes</span> <span class="n">row</span><span class="o">,</span> <span class="n">Column</span> <span class="n">col</span><span class="o">)</span> <span class="o">{</span>
+
+        <span class="c1">// create recording transaction (rtx)</span>
+        <span class="n">RecordingTransactionBase</span> <span class="n">rtx</span> <span class="o">=</span> <span class="n">RecordingTransactionBase</span><span class="o">.</span><span class="na">wrap</span><span class="o">(</span><span class="n">tx</span><span class="o">);</span>
+        
+        <span class="c1">// use rtx to create a typed transaction &amp; perform operations</span>
+        <span class="n">TypedTransactionBase</span> <span class="n">ttx</span> <span class="o">=</span> <span class="n">TYPEL</span><span class="o">.</span><span class="na">wrap</span><span class="o">(</span><span class="n">rtx</span><span class="o">);</span>
+        <span class="kt">int</span> <span class="n">count</span> <span class="o">=</span> <span class="n">ttx</span><span class="o">.</span><span class="na">get</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"counter1"</span><span class="o">).</span><span class="na">toInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
+        <span class="n">ttx</span><span class="o">.</span><span class="na">mutate</span><span class="o">().</span><span class="na">row</span><span class="o">(</span><span class="n">row</span><span class="o">).</span><span class="na">fam</span><span class="o">(</span><span class="s">"meta"</span><span class="o">).</span><span class="na">qual</span><span class="o">(</span><span class="s">"counter1"</span><span class="o">).</span><span class="na">set</span><span class="o">(</span><span class="n">count</span><span class="o">+</span><span class="mi">1</span><span class="o">);</span>
+        
+        <span class="c1">// when finished performing operations, retrieve transaction log</span>
+        <span class="n">TxLog</span> <span class="n">txLog</span> <span class="o">=</span> <span class="n">rtx</span><span class="o">.</span><span class="na">getTxLog</span><span class="o">()</span>
+
+        <span class="c1">// add txLog to exportQueue if not empty</span>
+        <span class="k">if</span> <span class="o">(!</span><span class="n">txLog</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">())</span> <span class="o">{</span>
+          <span class="c1">//do not pass rtx to exportQueue.add()</span>
+          <span class="n">exportQueue</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">tx</span><span class="o">,</span> <span class="n">row</span><span class="o">,</span> <span class="n">txLog</span><span class="o">)</span>
+        <span class="o">}</span>
+    <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+  </article>
+
+</div>
+
+          </div>
+      </div>
+      <hr>
+      <div class="row footer">
+        <div class="col-sm-12 text-center">
+          <div class="center-block">
+          <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+          Copyright &copy; 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache&nbsp;License,&nbsp;Version&nbsp;2.0</a>
+          </div>
+        </div>
+      </div>
+    </div>
+    <script src="/javascripts/jquery.min.js"></script>
+    <script src="/javascripts/bootstrap.min.js"></script>
+    <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+  ga('create', 'UA-55360307-1', 'auto');
+  ga('send', 'pageview');
+
+</script>
+
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/f6125fa2/docs/fluo-recipes/1.0.0-incubating/row-hasher/index.html
----------------------------------------------------------------------
diff --git a/docs/fluo-recipes/1.0.0-incubating/row-hasher/index.html b/docs/fluo-recipes/1.0.0-incubating/row-hasher/index.html
new file mode 100644
index 0000000..04245f6
--- /dev/null
+++ b/docs/fluo-recipes/1.0.0-incubating/row-hasher/index.html
@@ -0,0 +1,235 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <link rel="stylesheet" href="/stylesheets/fluo.css">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+    <link rel="canonical" href="https://fluo.apache.org//docs/fluo-recipes/1.0.0-incubating/row-hasher/">
+    <link rel="icon" type="image/png" href="/resources/favicon.png">
+    <title>Row hash prefix recipe | Apache Fluo</title>
+    <!--[if lt IE 9]>
+      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>
+    <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+      <div class="container">
+        <div class="navbar-header">
+          <div class="navbar-toggle-wrapper visible-xs">
+            <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+            </button>
+          </div>
+          <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+        </div>
+        <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+          <ul class="navbar-nav nav">
+            <li><a href="/release/">Releases</a></li>
+            <li><a href="/tour/">Tour</a></li>
+            <li><a href="/docs/">Docs</a></li>
+            <li><a href="/api/">API</a></li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/getinvolved/">Get Involved</a></li>
+                <li><a href="/news/">News Archive</a></li>
+                <li><a href="/people/">People</a></li>
+                <li><a href="/related-projects/">Related Projects</a></li>
+                <li><a href="/poweredby/">Powered By</a></li>
+              </ul>
+            </li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/how-to-contribute/">How To Contribute</a></li>
+                <li><a href="/release-process/">Release Process</a></li>
+              </ul>
+            </li>
+          </ul>
+          <ul class="navbar-nav nav navbar-right">
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="https://www.apache.org">Apache Homepage</a></li>
+                <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+                <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+                <li><a href="https://www.apache.org/security">Security</a></li>
+                <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+                <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+              </ul>
+            </li>
+          </ul>
+        </div>
+      </div>
+    </div>
+    <div class="container">
+      <div class="row">
+          <div class="col-sm-12">
+            <div class="fluo-doc">
+
+  
+
+  
+
+  <header class="post-header">
+    <h2 class="post-title">Row hash prefix recipe</h2>
+  </header>
+
+  <article id="page-content">
+    <h2 id="background">Background</h2>
+
+<p>Transactions are implemented in Fluo using conditional mutations.  Conditional
+mutations require server side processing on tservers.  If data is not spread
+evenly, it can cause some tservers to execute more conditional mutations than
+others.  These tservers doing more work can become a bottleneck.  Most real
+world data is not uniform and can cause this problem.</p>
+
+<p>Before the Fluo <a href="https://github.com/fluo-io/webindex">Webindex example</a> started using this recipe it suffered
+from this problem.  The example was using reverse dns encoded URLs for row keys
+like <code class="highlighter-rouge">p:com.cnn/story1.html</code>.  This made certain portions of the table more
+popular, which in turn made some tservers do much more work.  This uneven
+distribution of work lead to lower throughput and uneven performance.  Using
+this recipe made those problems go away.</p>
+
+<h2 id="solution">Solution</h2>
+
+<p>This recipe provides code to help add a hash of the row as a prefix of the row.
+Using this recipe rows are structured like the following.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>&lt;prefix&gt;:&lt;fixed len row hash&gt;:&lt;user row&gt;
+</code></pre>
+</div>
+
+<p>The recipe also provides code to help generate split points and configure
+balancing of the prefix.</p>
+
+<h2 id="example-use">Example Use</h2>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">org.apache.fluo.api.config.FluoConfiguration</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.data.Bytes</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.recipes.core.data.RowHasher</span><span class="o">;</span>
+
+<span class="kd">public</span> <span class="kd">class</span> <span class="nc">RowHasherExample</span> <span class="o">{</span>
+
+
+  <span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">RowHasher</span> <span class="n">PAGE_ROW_HASHER</span> <span class="o">=</span> <span class="k">new</span> <span class="n">RowHasher</span><span class="o">(</span><span class="s">"p"</span><span class="o">);</span>
+
+  <span class="c1">// Provide one place to obtain row hasher.</span>
+  <span class="kd">public</span> <span class="kd">static</span> <span class="n">RowHasher</span> <span class="nf">getPageRowHasher</span><span class="o">()</span> <span class="o">{</span>
+    <span class="k">return</span> <span class="n">PAGE_ROW_HASHER</span><span class="o">;</span>
+  <span class="o">}</span>
+
+  <span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="n">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="o">{</span>
+    <span class="n">RowHasher</span> <span class="n">pageRowHasher</span> <span class="o">=</span> <span class="n">getPageRowHasher</span><span class="o">();</span>
+
+    <span class="n">String</span> <span class="n">revUrl</span> <span class="o">=</span> <span class="s">"org.wikipedia/accumulo"</span><span class="o">;</span>
+
+    <span class="c1">// Add a hash prefix to the row. Use this hashedRow in your transaction</span>
+    <span class="n">Bytes</span> <span class="n">hashedRow</span> <span class="o">=</span> <span class="n">pageRowHasher</span><span class="o">.</span><span class="na">addHash</span><span class="o">(</span><span class="n">revUrl</span><span class="o">);</span>
+    <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"hashedRow      : "</span> <span class="o">+</span> <span class="n">hashedRow</span><span class="o">);</span>
+
+    <span class="c1">// Remove the prefix. This can be used by transactions dealing with the hashed row.</span>
+    <span class="n">Bytes</span> <span class="n">orig</span> <span class="o">=</span> <span class="n">pageRowHasher</span><span class="o">.</span><span class="na">removeHash</span><span class="o">(</span><span class="n">hashedRow</span><span class="o">);</span>
+    <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"orig           : "</span> <span class="o">+</span> <span class="n">orig</span><span class="o">);</span>
+
+
+    <span class="c1">// Generate table optimizations for the recipe. This can be called when setting up an</span>
+    <span class="c1">// application that uses a hashed row.</span>
+    <span class="kt">int</span> <span class="n">numTablets</span> <span class="o">=</span> <span class="mi">20</span><span class="o">;</span>
+
+    <span class="c1">// The following code would normally be called before initializing Fluo. This code</span>
+    <span class="c1">// registers table optimizations for your prefix+hash.</span>
+    <span class="n">FluoConfiguration</span> <span class="n">conf</span> <span class="o">=</span> <span class="k">new</span> <span class="n">FluoConfiguration</span><span class="o">();</span>
+    <span class="n">RowHasher</span><span class="o">.</span><span class="na">configure</span><span class="o">(</span><span class="n">conf</span><span class="o">,</span> <span class="n">PAGE_ROW_HASHER</span><span class="o">.</span><span class="na">getPrefix</span><span class="o">(),</span> <span class="n">numTablets</span><span class="o">);</span>
+
+    <span class="c1">// Normally you would not call the following code, it would be called automatically for you by</span>
+    <span class="c1">// TableOperations.optimizeTable(). Calling this code here to show what table optimization will</span>
+    <span class="c1">// be generated.</span>
+    <span class="n">TableOptimizations</span> <span class="n">tableOptimizations</span> <span class="o">=</span> <span class="k">new</span> <span class="n">RowHasher</span><span class="o">.</span><span class="na">Optimizer</span><span class="o">()</span>
+        <span class="o">.</span><span class="na">getTableOptimizations</span><span class="o">(</span><span class="n">PAGE_ROW_HASHER</span><span class="o">.</span><span class="na">getPrefix</span><span class="o">(),</span> <span class="n">conf</span><span class="o">.</span><span class="na">getAppConfiguration</span><span class="o">());</span>
+    <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Balance config : "</span> <span class="o">+</span> <span class="n">tableOptimizations</span><span class="o">.</span><span class="na">getTabletGroupingRegex</span><span class="o">());</span>
+    <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"Splits         : "</span><span class="o">);</span>
+    <span class="n">tableOptimizations</span><span class="o">.</span><span class="na">getSplits</span><span class="o">().</span><span class="na">forEach</span><span class="o">(</span><span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">::</span><span class="n">println</span><span class="o">);</span>
+    <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">();</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>The example program above prints the following.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>hashedRow      : p:1yl0:org.wikipedia/accumulo
+orig           : org.wikipedia/accumulo
+Balance config : (\Qp:\E).*
+Splits         : 
+p:1sst
+p:3llm
+p:5eef
+p:7778
+p:9001
+p:assu
+p:clln
+p:eeeg
+p:g779
+p:i002
+p:jssv
+p:lllo
+p:neeh
+p:p77a
+p:r003
+p:sssw
+p:ullp
+p:weei
+p:y77b
+p:~
+</code></pre>
+</div>
+
+<p>The split points are used to create tablets in the Accumulo table used by Fluo.
+Data and computation will spread very evenly across these tablets.  The
+Balancing config will spread the tablets evenly across the tablet servers,
+which will spread the computation evenly. See the <a href="/docs/fluo-recipes/1.0.0-incubating/table-optimization/">table optimizations</a>
+documentation for information on how to apply the optimizations.</p>
+
+
+  </article>
+
+</div>
+
+          </div>
+      </div>
+      <hr>
+      <div class="row footer">
+        <div class="col-sm-12 text-center">
+          <div class="center-block">
+          <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+          Copyright &copy; 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache&nbsp;License,&nbsp;Version&nbsp;2.0</a>
+          </div>
+        </div>
+      </div>
+    </div>
+    <script src="/javascripts/jquery.min.js"></script>
+    <script src="/javascripts/bootstrap.min.js"></script>
+    <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+  ga('create', 'UA-55360307-1', 'auto');
+  ga('send', 'pageview');
+
+</script>
+
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/f6125fa2/docs/fluo-recipes/1.0.0-incubating/serialization/index.html
----------------------------------------------------------------------
diff --git a/docs/fluo-recipes/1.0.0-incubating/serialization/index.html b/docs/fluo-recipes/1.0.0-incubating/serialization/index.html
new file mode 100644
index 0000000..0169b06
--- /dev/null
+++ b/docs/fluo-recipes/1.0.0-incubating/serialization/index.html
@@ -0,0 +1,188 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <link rel="stylesheet" href="/stylesheets/fluo.css">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+    <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+    <link rel="canonical" href="https://fluo.apache.org//docs/fluo-recipes/1.0.0-incubating/serialization/">
+    <link rel="icon" type="image/png" href="/resources/favicon.png">
+    <title>Serializing Data | Apache Fluo</title>
+    <!--[if lt IE 9]>
+      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>
+    <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+      <div class="container">
+        <div class="navbar-header">
+          <div class="navbar-toggle-wrapper visible-xs">
+            <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+              <span class="icon-bar"></span>
+            </button>
+          </div>
+          <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+        </div>
+        <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+          <ul class="navbar-nav nav">
+            <li><a href="/release/">Releases</a></li>
+            <li><a href="/tour/">Tour</a></li>
+            <li><a href="/docs/">Docs</a></li>
+            <li><a href="/api/">API</a></li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/getinvolved/">Get Involved</a></li>
+                <li><a href="/news/">News Archive</a></li>
+                <li><a href="/people/">People</a></li>
+                <li><a href="/related-projects/">Related Projects</a></li>
+                <li><a href="/poweredby/">Powered By</a></li>
+              </ul>
+            </li>
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="/how-to-contribute/">How To Contribute</a></li>
+                <li><a href="/release-process/">Release Process</a></li>
+              </ul>
+            </li>
+          </ul>
+          <ul class="navbar-nav nav navbar-right">
+            <li class="dropdown">
+              <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+              <ul class="dropdown-menu">
+                <li><a href="https://www.apache.org">Apache Homepage</a></li>
+                <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+                <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+                <li><a href="https://www.apache.org/security">Security</a></li>
+                <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+                <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+              </ul>
+            </li>
+          </ul>
+        </div>
+      </div>
+    </div>
+    <div class="container">
+      <div class="row">
+          <div class="col-sm-12">
+            <div class="fluo-doc">
+
+  
+
+  
+
+  <header class="post-header">
+    <h2 class="post-title">Serializing Data</h2>
+  </header>
+
+  <article id="page-content">
+    <p>Various Fluo Recipes deal with POJOs and need to serialize them.  The
+serialization mechanism is configurable and defaults to using <a href="https://github.com/EsotericSoftware/kryo">Kryo</a>.</p>
+
+<h2 id="custom-serialization">Custom Serialization</h2>
+
+<p>In order to use a custom serialization method, two steps need to be taken.  The
+first step is to implement <a href="https://static.javadoc.io/org.apache.fluo/fluo-recipes-core/1.0.0-incubating/org/apache/fluo/recipes/core/serialization/SimpleSerializer.html">SimpleSerializer</a>.  The second step is to
+configure Fluo Recipes to use the custom implementation.  This needs to be done
+before initializing Fluo.  Below is an example of how to do this.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code>  <span class="n">FluoConfiguration</span> <span class="n">fluoConfig</span> <span class="o">=</span> <span class="o">...;</span>
+  <span class="c1">//assume MySerializer implements SimpleSerializer</span>
+  <span class="n">SimpleSerializer</span><span class="o">.</span><span class="na">setSetserlializer</span><span class="o">(</span><span class="n">fluoConfig</span><span class="o">,</span> <span class="n">MySerializer</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
+  <span class="c1">//initialize Fluo using fluoConfig</span>
+</code></pre>
+</div>
+
+<h2 id="kryo-factory">Kryo Factory</h2>
+
+<p>If using the default Kryo serializer implementation, then creating a
+KryoFactory implementation can lead to smaller serialization size.  When Kryo
+serializes an object graph, it will by default include the fully qualified
+names of the classes in the serialized data.  This can be avoided by
+<a href="https://github.com/EsotericSoftware/kryo#registration">registering classes</a> that will be serialized.  Registration is done by
+creating a KryoFactory and then configuring Fluo Recipes to use it.   The
+example below shows how to do this.</p>
+
+<p>For example assume the POJOs named <code class="highlighter-rouge">Node</code> and <code class="highlighter-rouge">Edge</code> will be serialized and
+need to be registered with Kryo.  This could be done by creating a KryoFactory
+like the following.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code>
+<span class="kn">package</span> <span class="n">com</span><span class="o">.</span><span class="na">foo</span><span class="o">;</span>
+
+<span class="kn">import</span> <span class="nn">com.esotericsoftware.kryo.Kryo</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">com.esotericsoftware.kryo.pool.KryoFactory</span><span class="o">;</span>
+
+<span class="kn">import</span> <span class="nn">com.foo.data.Edge</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">com.foo.data.Node</span><span class="o">;</span>
+
+<span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyKryoFactory</span> <span class="kd">implements</span> <span class="n">KryoFactory</span> <span class="o">{</span>
+  <span class="nd">@Override</span>
+  <span class="kd">public</span> <span class="n">Kryo</span> <span class="nf">create</span><span class="o">()</span> <span class="o">{</span>
+    <span class="n">Kryo</span> <span class="n">kryo</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Kryo</span><span class="o">();</span>
+    
+    <span class="c1">//Explicitly assign each class a unique id here to ensure its stable over</span>
+    <span class="c1">//time and in different environments with different dependencies.</span>
+    <span class="n">kryo</span><span class="o">.</span><span class="na">register</span><span class="o">(</span><span class="n">Node</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="mi">9</span><span class="o">);</span>
+    <span class="n">kryo</span><span class="o">.</span><span class="na">register</span><span class="o">(</span><span class="n">Edge</span><span class="o">.</span><span class="na">class</span><span class="o">,</span> <span class="mi">10</span><span class="o">);</span>
+    
+    <span class="c1">//instruct kryo that these are the only classes we expect to be serialized</span>
+    <span class="n">kryo</span><span class="o">.</span><span class="na">setRegistrationRequired</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
+    
+    <span class="k">return</span> <span class="n">kryo</span><span class="o">;</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>Fluo Recipes must be configured to use this factory.  The following code shows
+how to do this.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code>  <span class="n">FluoConfiguration</span> <span class="n">fluoConfig</span> <span class="o">=</span> <span class="o">...;</span>
+  <span class="n">KryoSimplerSerializer</span><span class="o">.</span><span class="na">setKryoFactory</span><span class="o">(</span><span class="n">fluoConfig</span><span class="o">,</span> <span class="n">MyKryoFactory</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
+  <span class="c1">//initialize Fluo using fluoConfig</span>
+</code></pre>
+</div>
+
+
+  </article>
+
+</div>
+
+          </div>
+      </div>
+      <hr>
+      <div class="row footer">
+        <div class="col-sm-12 text-center">
+          <div class="center-block">
+          <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+          Copyright &copy; 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache&nbsp;License,&nbsp;Version&nbsp;2.0</a>
+          </div>
+        </div>
+      </div>
+    </div>
+    <script src="/javascripts/jquery.min.js"></script>
+    <script src="/javascripts/bootstrap.min.js"></script>
+    <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+  ga('create', 'UA-55360307-1', 'auto');
+  ga('send', 'pageview');
+
+</script>
+
+  </body>
+</html>


Mime
View raw message