bookkeeper-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r988319 [39/40] - in /websites/staging/bookkeeper/trunk/content: ./ docs/master/ docs/r4.0.0/ docs/r4.1.0/ docs/r4.2.0/ docs/r4.2.1/ docs/r4.2.2/ docs/r4.2.3/ docs/r4.2.4/ docs/r4.3.0/ docs/r4.3.1/ docs/r4.3.2/ docs/r4.4.0/ docs/r4.4.0/apid...
Date Sun, 15 May 2016 21:40:28 GMT
Added: websites/staging/bookkeeper/trunk/content/docs/r4.4.0/bookkeeperStream.html
==============================================================================
--- websites/staging/bookkeeper/trunk/content/docs/r4.4.0/bookkeeperStream.html (added)
+++ websites/staging/bookkeeper/trunk/content/docs/r4.4.0/bookkeeperStream.html Sun May 15 21:40:24 2016
@@ -0,0 +1,259 @@
+<!DOCTYPE html>
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE- 2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<html>
+  <head>
+    <title>Apache BookKeeper - Streaming with BookKeeper</title>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <!-- Bootstrap -->
+    <link href="/css/bootstrap.min.css" rel="stylesheet">
+    <link href="/css/bootstrap-responsive.min.css" rel="stylesheet">
+    <link href="/css/styles.css" rel="stylesheet">
+  </head>
+  <body>
+    <header class="navbar navbar-inverse navbar-static-top" role="banner">
+      <div class="container">
+    	<div class="navbar-header hidden-xs hidden-sm">
+    	  <a class="navbar-brand navbar-logo" href="/"><img class="img-responsive" src="/img/bookkeeper_blk40.png" alt="Bookkeeper Logo" /></a>
+    	</div>
+    	<div class="navbar-header">
+    	  <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
+            <span class="sr-only">Toggle navigation</span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+    	  </button>
+    	  <a class="navbar-brand" href="/">Apache BookKeeper</a>
+    	</div>
+    	<nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
+    	  <ul class="nav navbar-nav">
+    	    <li><a href="/releases.html">Download</a></li>
+
+    	    <li class="dropdown">
+    	      <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Documentation<span class="caret"></span></a>
+    	      <ul class="dropdown-menu" role="menu">
+		<li><a href="/docs/master">Latest (master)</a></li>
+		<li><ul>
+		    <li><a href="/docs/master/apidocs">Java API docs</a></li>
+		    <li><a href="/docs/master/bookkeeperTutorial.html">Tutorial</a></li>
+		    <li><a href="/docs/master/bookkeeperConfig.html">Admin guide</a></li>
+		</ul><li>
+                <li><a href="/docs/r4.4.0">Release 4.4.0</a></li>
+                <li class="divider"></li>
+                <li>Older releases</li>
+                <li><a href="/docs/r4.3.2">Release 4.3.2</a></li>
+                <li><a href="/docs/r4.3.1">Release 4.3.1</a></li>
+                <li><a href="/docs/r4.3.0">Release 4.3.0</a></li>
+                <li><a href="/docs/r4.2.4">Release 4.2.4</a></li>
+                <li><a href="/docs/r4.2.3">Release 4.2.3</a></li>
+                <li><a href="/docs/r4.2.2">Release 4.2.2</a></li>
+                <li><a href="/docs/r4.2.1">Release 4.2.1</a></li>
+                <li><a href="/docs/r4.2.0">Release 4.2.0</a></li>
+                <li><a href="/docs/r4.1.0">Release 4.1.0</a></li>
+                <li><a href="/docs/r4.0.0">Release 4.0.0</a></li>
+              </ul>
+            </li>
+            
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Get Involved<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/lists.html">Mailing Lists</a></li>
+                <li><a href="/irc.html">IRC</a></li>
+                <li><a href="/svn.html">Version Control</a></li>
+                <li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+              </ul>
+            </li>
+
+            <li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+            <!--<li><a href="#">Hedwig</a></li>//-->
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project Info<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/credits.html">Who are we?</a></li>
+                <li><a href="/bylaws.html">Bylaws</a></li>
+                <li><a href="http://www.apache.org/licenses/">License</a></li>
+                <li class="divider"></li>
+                <li><a href="/privacy.html">Privacy Policy</a></li>
+                <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsership</a></li>
+                <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+              </ul>
+            </li>
+          </ul>
+          <script>
+            (function() {
+            var cx = '017580107654524087317:iqnsyimpydg';
+            var gcse = document.createElement('script');
+            gcse.type = 'text/javascript';
+            gcse.async = true;
+            gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+            '//www.google.com/cse/cse.js?cx=' + cx;
+            var s = document.getElementsByTagName('script')[0];
+            s.parentNode.insertBefore(gcse, s);
+            })();
+          </script>
+          
+          <div class="navbar-form navbar-right visible-lg" id="googlebox">
+            <gcse:searchbox-only></gcse:searchbox-only>
+          </div>
+        </nav>
+      </div>
+    </header>
+    <div class="container">
+
+ <h1>Abstract</h1>
+
+<p>This guide contains detailed information about using how to stream bytes on top of BookKeeper. It essentially motivates and discusses the basic stream operations currently supported.</p>
+
+<h1>Summary</h1>
+
+<p>When using the BookKeeper <span class="caps">API, </span>an application has to split the data to write into entries, each entry being a byte array. This is natural for many applications. For example, when using BookKeeper for write-ahead logging, an application typically wants to write the modifications corresponding to a command or a transaction. Some other applications, however, might not have a natural boundary for entries, and may prefer to write and read streams of bytes. This is exactly the purpose of the stream <span class="caps">API </span>we have implemented on top of BookKeeper. </p>
+
+<p>The stream <span class="caps">API </span>is implemented in the package <code>Streaming</code> , and it contains two main classes: <code>LedgerOutputStream</code> and  <code>LedgerInputStream</code> . The class names are indicative of what they do. </p>
+
+<h1>Writing a stream of bytes</h1>
+
+<p>Class <code>LedgerOutputStream</code> implements two constructors and five public methods: </p>
+
+<p> <code>public LedgerOutputStream(LedgerHandle lh) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+</ul>
+
+
+<p> <code>public LedgerOutputStream(LedgerHandle lh, int size) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+<li> <code>size</code> is the size of the byte buffer to store written bytes before flushing. </li>
+</ul>
+
+
+<p> <em>Closing a stream.</em> This call closes the stream by flushing the write buffer. </p>
+
+<p> <code>public void close() </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <em>Flushing a stream.</em> This call essentially flushes the write buffer. </p>
+
+<p> <code>public synchronized void flush() </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <em>Writing bytes.</em> There are three calls for writing bytes to a stream. </p>
+
+<p> <code>public synchronized void write(byte[] b) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is an array of bytes to write. </li>
+</ul>
+
+
+<p> <code>public synchronized void write(byte[] b, int off, int len) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is an array of bytes to write. </li>
+<li> <code>off</code> is a buffer offset. </li>
+<li> <code>len</code> is the length to write. </li>
+</ul>
+
+
+<p> <code>public synchronized void write(int b) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> contains a byte to write. The method writes the least significant byte of the integer four bytes. </li>
+</ul>
+
+
+<h1>Reading a stream of bytes</h1>
+
+<p>Class <code>LedgerOutputStream</code> implements two constructors and four public methods: </p>
+
+<p> <code>public LedgerInputStream(LedgerHandle lh) throws BKException, InterruptedException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+</ul>
+
+
+<p> <code>public LedgerInputStream(LedgerHandle lh, int size) throws BKException, InterruptedException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+<li> <code>size</code> is the size of the byte buffer to store bytes that the application will eventually read. </li>
+</ul>
+
+
+<p> <em>Closing.</em> There is one call to close an input stream, but the call is currently empty and the application is responsible for closing the ledger handle. </p>
+
+<p> <code>public void close() </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <em>Reading.</em> There are three calls to read from the stream. </p>
+
+<p> <code>public synchronized int read() throws IOException </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <code>public synchronized int read(byte[] b) throws IOException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is a byte array to write to. </li>
+</ul>
+
+
+<p> <code>public synchronized int read(byte[] b, int off, int len) throws IOException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is a byte array to write to. </li>
+<li> <code>off</code> is an offset for byte array <code>b</code> . </li>
+<li> <code>len</code> is the length in bytes to write to <code>b</code> . </li>
+</ul>
+
+    </div>
+    <footer class="footer">
+      <div class="container">
+        <p class="text-muted">Copyright &copy; 2014 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.<br/>
+	Apache BookKeeper, BookKeeper, Apache, Apache ZooKeeper, ZooKeeper, the Apache feather logo, and the Apache BookKeeper project logo are trademarks of The Apache Software Foundation.</p>
+      </div>
+    </footer>
+
+    <script src="http://code.jquery.com/jquery.js"></script>
+    <script src="/js/bootstrap.min.js"></script>
+  </body>
+</html>

Added: websites/staging/bookkeeper/trunk/content/docs/r4.4.0/bookkeeperTutorial.html
==============================================================================
--- websites/staging/bookkeeper/trunk/content/docs/r4.4.0/bookkeeperTutorial.html (added)
+++ websites/staging/bookkeeper/trunk/content/docs/r4.4.0/bookkeeperTutorial.html Sun May 15 21:40:24 2016
@@ -0,0 +1,740 @@
+<!DOCTYPE html>
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE- 2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<html>
+  <head>
+    <title>Apache BookKeeper - Bookkeeper Client tutorial</title>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <!-- Bootstrap -->
+    <link href="/css/bootstrap.min.css" rel="stylesheet">
+    <link href="/css/bootstrap-responsive.min.css" rel="stylesheet">
+    <link href="/css/styles.css" rel="stylesheet">
+  </head>
+  <body>
+    <header class="navbar navbar-inverse navbar-static-top" role="banner">
+      <div class="container">
+    	<div class="navbar-header hidden-xs hidden-sm">
+    	  <a class="navbar-brand navbar-logo" href="/"><img class="img-responsive" src="/img/bookkeeper_blk40.png" alt="Bookkeeper Logo" /></a>
+    	</div>
+    	<div class="navbar-header">
+    	  <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
+            <span class="sr-only">Toggle navigation</span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+    	  </button>
+    	  <a class="navbar-brand" href="/">Apache BookKeeper</a>
+    	</div>
+    	<nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
+    	  <ul class="nav navbar-nav">
+    	    <li><a href="/releases.html">Download</a></li>
+
+    	    <li class="dropdown">
+    	      <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Documentation<span class="caret"></span></a>
+    	      <ul class="dropdown-menu" role="menu">
+		<li><a href="/docs/master">Latest (master)</a></li>
+		<li><ul>
+		    <li><a href="/docs/master/apidocs">Java API docs</a></li>
+		    <li><a href="/docs/master/bookkeeperTutorial.html">Tutorial</a></li>
+		    <li><a href="/docs/master/bookkeeperConfig.html">Admin guide</a></li>
+		</ul><li>
+                <li><a href="/docs/r4.4.0">Release 4.4.0</a></li>
+                <li class="divider"></li>
+                <li>Older releases</li>
+                <li><a href="/docs/r4.3.2">Release 4.3.2</a></li>
+                <li><a href="/docs/r4.3.1">Release 4.3.1</a></li>
+                <li><a href="/docs/r4.3.0">Release 4.3.0</a></li>
+                <li><a href="/docs/r4.2.4">Release 4.2.4</a></li>
+                <li><a href="/docs/r4.2.3">Release 4.2.3</a></li>
+                <li><a href="/docs/r4.2.2">Release 4.2.2</a></li>
+                <li><a href="/docs/r4.2.1">Release 4.2.1</a></li>
+                <li><a href="/docs/r4.2.0">Release 4.2.0</a></li>
+                <li><a href="/docs/r4.1.0">Release 4.1.0</a></li>
+                <li><a href="/docs/r4.0.0">Release 4.0.0</a></li>
+              </ul>
+            </li>
+            
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Get Involved<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/lists.html">Mailing Lists</a></li>
+                <li><a href="/irc.html">IRC</a></li>
+                <li><a href="/svn.html">Version Control</a></li>
+                <li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+              </ul>
+            </li>
+
+            <li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+            <!--<li><a href="#">Hedwig</a></li>//-->
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project Info<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/credits.html">Who are we?</a></li>
+                <li><a href="/bylaws.html">Bylaws</a></li>
+                <li><a href="http://www.apache.org/licenses/">License</a></li>
+                <li class="divider"></li>
+                <li><a href="/privacy.html">Privacy Policy</a></li>
+                <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsership</a></li>
+                <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+              </ul>
+            </li>
+          </ul>
+          <script>
+            (function() {
+            var cx = '017580107654524087317:iqnsyimpydg';
+            var gcse = document.createElement('script');
+            gcse.type = 'text/javascript';
+            gcse.async = true;
+            gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+            '//www.google.com/cse/cse.js?cx=' + cx;
+            var s = document.getElementsByTagName('script')[0];
+            s.parentNode.insertBefore(gcse, s);
+            })();
+          </script>
+          
+          <div class="navbar-form navbar-right visible-lg" id="googlebox">
+            <gcse:searchbox-only></gcse:searchbox-only>
+          </div>
+        </nav>
+      </div>
+    </header>
+    <div class="container">
+
+ <p>This tutorial aims to show you how to build a replicated distributed system using Bookkeeper as the replicated log. Before we start, you will need to have a bookkeeper cluster up and running. You can download the bookkeeper distribution at <a href="http://bookkeeper.apache.org/releases.html">http://bookkeeper.apache.org/releases.html</a>. The binary distribution, bookkeeper-server-4.x.x-bin.tar.gz, will be sufficient for the tutorial.<br />
+This tutorial does not cover the setup of a distributed cluster, but you can run a local cluster on your machine by running:</p>
+
+
+
+<pre>
+$ bookkeeper-server/bin/bookkeeper localbookie 6
+</pre>
+
+
+
+<p>This will start up a local zookeeper instance with 6 bookie servers, as bookkeeper storage servers are known. Any data written to this cluster will be removed when you kill the process.</p>
+
+<p>The code for this tutorial is available at <a href="https://github.com/ivankelly/bookkeeper-tutorial/">https://github.com/ivankelly/bookkeeper-tutorial/</a>. Each section has a link with points to a tag for the completed code for that section.</p>
+
+<h1>The base application</h1>
+
+<p><a href="https://github.com/ivankelly/bookkeeper-tutorial/tree/basic">(full code)</a></p>
+
+<p>We have a dice application. It generates a new number between 1 and 6 every second. </p>
+
+
+
+<pre class="prettyprint">
+public class Dice {
+
+    Random r = new Random();
+
+    void playDice() throws InterruptedException {
+        while (true) {
+            Thread.sleep(1000);
+            System.out.println(&quot;Value = &quot; + (r.nextInt(6) + 1));
+        }
+    }
+
+    public static void main(String[] args) throws InterruptedException {
+        Dice d = new Dice();
+        d.playDice();
+    }
+}
+</pre>
+
+
+
+<p>Our goal is to have multiple instances of this application, possibly running on different machine, which each display the exact same sequence of numbers. If one the the instances crashes or becomes unable to communicate with the others in any way, it should still not diverge from the sequence of numbers. This tutorial will show you how to achieve this.</p>
+
+<p>To start, download the base application, compile and run it.</p>
+
+
+<pre>
+$ git clone https://github.com/ivankelly/bookkeeper-tutorial.git
+$ mvn package
+$ mvn exec:java -Dexec.mainClass=org.apache.bookkeeper.Dice
+[INFO] Scanning for projects...
+[INFO]                                                                         
+[INFO] ------------------------------------------------------------------------
+[INFO] Building tutorial 1.0-SNAPSHOT
+[INFO] ------------------------------------------------------------------------
+[INFO] 
+[INFO] --- exec-maven-plugin:1.3.2:java (default-cli) @ tutorial ---
+[WARNING] Warning: killAfter is now deprecated. Do you need it ? Please comment on MEXEC-6.
+Value = 4
+Value = 5
+Value = 3
+...
+...
+</pre>
+
+
+
+
+<h1>Leaders and followers (and a little bit of background)</h1>
+
+<p>To achieve this common view in multiple instances of the program, we need each instance to agree on what the next number in the sequence will be. For example, the instances must agree that 4 is the first number and 2 is the second number and 5 is the third number and so on. This is a difficult problem, especially in the case that any instance may go away at any time, and messages between the instances can be lost or reordered.</p>
+
+<p>Luckily, there are already algorithms to solve this. <a href="http://en.wikipedia.org/wiki/Paxos_%28computer_science%29">Paxos</a> is an abstract algorithm to implement this kind of agreement, while <a href="http://zookeeper.apache.org">Zab</a> and <a href="http://en.wikipedia.org/wiki/Raft_%28computer_science%29">Raft</a> are more practical protocols. <a href="https://www.youtube.com/watch?v=JEpsBg0AO6o">This video</a> gives a good overview about how these algorithms usually look. They all have a similar core.</p>
+
+<p>It would be possible to run the Paxos to agree on each number in the sequence. However, running Paxos each time can be expensive. What Zab and Raft do is that they use a Paxos-like algorithm to elect a leader. The leader then decides what the sequence of events should be, putting them in a log, which the other instances can then follow to maintain the same state as the leader.</p>
+
+<p>Bookkeeper provides the functionality for the second part of the protocol, allowing a leader to write events to a log and have multiple followers tailing the log. However, bookkeeper does not do leader election. You will need a zookeeper or raft instance for that purpose.</p>
+
+<h2>Why not just use zookeeper for everything?</h2>
+
+<p>There are a number of reasons:</p>
+
+<p> 1. Zookeeper's log is only exposed through a tree like interface. It can be hard to shoehorn your application into this. <br />
+ 2. A zookeeper ensemble of multiple machines is limited to one log. You may want one log per resource, which will become expensive very quickly.<br />
+ 3. Adding extra machines to a zookeeper ensemble does not increase capacity nor throughput.</p>
+
+<p>Bookkeeper can be viewed as a means of exposing zookeeper's replicated log to applications in a scalable fashion. However, we still use zookeeper to maintain consistency guarantees.</p>
+
+<p><b>TL;DR You need to elect a leader instance</b></p>
+
+<h1>Electing a leader</h1>
+
+<p><a href="https://github.com/ivankelly/bookkeeper-tutorial/tree/election">(full code)</a></p>
+
+<p>We'll use zookeeper to elect a leader. A zookeeper instance will have started locally when you started the localbookie application above. To verify it's running, run the following command.</p>
+
+
+
+<pre>
+$ echo stat | nc localhost 2181
+Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
+Clients:
+ /127.0.0.1:59343[1](queued=0,recved=40,sent=41)
+ /127.0.0.1:49354[1](queued=0,recved=11,sent=11)
+ /127.0.0.1:49361[0](queued=0,recved=1,sent=0)
+ /127.0.0.1:59344[1](queued=0,recved=38,sent=39)
+ /127.0.0.1:59345[1](queued=0,recved=38,sent=39)
+ /127.0.0.1:59346[1](queued=0,recved=38,sent=39)
+
+Latency min/avg/max: 0/0/23
+Received: 167
+Sent: 170
+Connections: 6
+Outstanding: 0
+Zxid: 0x11
+Mode: standalone
+Node count: 16
+</pre>
+
+
+
+<p>To interact with zookeeper, we'll use the <a href="https://curator.apache.org/">Curator</a> client rather than the stock zookeeper client. Getting things right with the zookeeper client can be tricky, and curator removes a lot of the pointy corners for you. In fact, curator even provides a leader election recipe, so we need to do very little work to get leader election in our application.</p>
+
+
+
+<pre class="prettyprint">
+public class Dice extends LeaderSelectorListenerAdapter implements Closeable {
+
+    final static String ZOOKEEPER_SERVER = &quot;127.0.0.1:2181&quot;;
+    final static String ELECTION_PATH = &quot;/dice-elect&quot;;
+
+    ...
+
+    Dice() throws InterruptedException {
+        curator = CuratorFrameworkFactory.newClient(ZOOKEEPER_SERVER,
+                2000, 10000, new ExponentialBackoffRetry(1000, 3));
+        curator.start();
+        curator.blockUntilConnected();
+
+        leaderSelector = new LeaderSelector(curator, ELECTION_PATH, this);
+        leaderSelector.autoRequeue();
+        leaderSelector.start();
+    }
+</pre>
+
+
+
+<p>In the constructor for Dice, we need to create the curator client. We specify four things when creating the client, the location of the zookeeper service, the session timeout, the connect timeout and the retry policy.</p>
+
+<p>The session timeout is a zookeeper concept. If the zookeeper server doesn't hear anything from the client for this amount of time, any leases which the client holds will be timed out. This is important in leader election. For leader election, the curator client will take a lease on <span class="caps">ELECTION</span>_PATH. The first instance to take the lease will become leader and the rest will become followers. However, their claim on the lease will remain in the cue. If the first instance then goes away, due to a crash etc., its session will timeout. Once the session times out, the lease will be released and the next instance in the queue will become the leader. The call to <mark>autoRequeue()</mark> will make the client queue itself again if it loses the lease for some other reason, such as if it was still alive, but it a garbage collection cycle caused it to lose its session, and thereby its lease. I've set the lease to be quite low so that when we test out leader election, t
 ransitions will be quite quick. The optimum length for session timeout depends very much on the use case. The other parameters are the connection timeout, i.e. the amount of time it will spend trying to connect to a zookeeper server before giving up, and the retry policy. The retry policy specifies how the client should respond to transient errors, such as connection loss. Operations that fail with transient errors can be retried, and this argument specifies how often the retries should occur.</p>
+
+<p>Finally, you'll have noticed that Dice now extends <mark>LeaderSelectorListenerAdapter</mark> and implements <mark>Closeable</mark>. <mark>Closeable</mark> is there to close the resource we have initialized in the constructor, the curator client and the <mark>leaderSelector</mark>. <mark>LeaderSelectorListenerAdapter</mark> is a callback that the <mark>leaderSelector</mark> uses to notify the instance that it is now the leader. It is passed as the third argument to the <mark>LeaderSelector</mark> constructor.</p>
+
+
+
+<pre class="prettyprint">
+    @Override
+    public void takeLeadership(CuratorFramework client)
+            throws Exception {
+        synchronized (this) {
+            leader = true;
+            try {
+                while (true) {
+                    this.wait();
+                }
+            } catch (InterruptedException ie) {
+                Thread.currentThread().interrupt();
+                leader = false;
+            }
+        }
+    }
+</pre>
+
+
+
+<p><mark>takeLeadership()</mark> is the callback called by <mark>LeaderSelector</mark> when the instance is leader. It should only return when the instance wants to give up leadership. In our case, we never do so we wait on the current object until we're interrupted. To signal to the rest of the program that we are leader we set a volatile boolean called leader to true. This is unset after we are interrupted.</p>
+
+
+
+<pre class="prettyprint">
+    void playDice() throws InterruptedException {
+        while (true) {
+            while (leader) {
+                Thread.sleep(1000);
+                System.out.println(&quot;Value = &quot; + (r.nextInt(6) + 1)
+                                   + &quot;, isLeader = &quot; + leader);
+            }
+        }
+    }
+</pre>
+
+
+
+<p>Finally we modify <mark>playDice()</mark> to only generate random numbers when it is the leader.</p>
+
+<p>Run two instances of the program in two different terminals. You'll see that one becomes leader and prints numbers and the other just sits there.</p>
+
+<p>Now stop the leader using Control-Z. This will pause the process, but it won't kill it. You will be dropped back to the shell in that terminal. After a couple of seconds, the session timeout, you will see that the other instance has become the leader. Zookeeper will guarantee that only one instance is selected as leader at any time.</p>
+
+<p>Now go back to the shell that the original leader was on and wake up the process using fg. You'll see something like the following:</p>
+
+
+
+<pre>
+...
+...
+Value = 4, isLeader = true
+Value = 4, isLeader = true
+^Z
+[1]+  Stopped                 mvn exec:java -Dexec.mainClass=org.apache.bookkeeper.Dice
+$ fg
+mvn exec:java -Dexec.mainClass=org.apache.bookkeeper.Dice
+Value = 3, isLeader = true
+Value = 1, isLeader = false
+</pre>
+
+
+
+<p>Whats this!?! The other instance is leader, but this instance first of all thinks it is leader and generates a number, and then generates a number even though it knows it is not leader. In fact this is perfectly natural. The leader election happens on zookeeper, but it takes time changes in the leader to be propagated to all instances. So a race occurs where an instance thinks it is the leader while zookeeper thinks otherwise.</p>
+
+<p>To solve this problem we need to some way to prevent previous leaders from continuing to think they are the leader. Sending a message to the previous leader isn't an option. Messages may get lost or delayed or the previous leader may be temporarily down. Another way is to use a shared log. All updates are written to the shared log before being applied. A new leader can tell this log to block writes from previous leaders. This is exactly what bookkeeper does!</p>
+
+<h1>Writing to the log</h1>
+
+<p><a href="https://github.com/ivankelly/bookkeeper-tutorial/tree/storing">(full code)</a></p>
+
+<p>Before we get into the business of blocking previous leaders from writing we need to first implement the logic for writing to the log.</p>
+
+
+
+<pre class="prettyprint">
+    Dice() throws Exception {
+    	...
+
+        ClientConfiguration conf = new ClientConfiguration()
+            .setZkServers(ZOOKEEPER_SERVER).setZkTimeout(30000);
+        bookkeeper = new BookKeeper(conf);
+    }
+
+</pre>
+
+
+
+<p>We construct the bookkeeper client in the <mark>Dice</mark> constructor and configure the zookeeper server and zookeeper session timeout that it should use. The zookeeper session timeout can be quite large for bookkeeper, as it doesn't use anything that depends on the session timeout logic. The bookkeeper client should also be closed in <mark>Dice#close()</mark>.</p>
+
+
+
+<pre class="prettyprint">
+    void lead() throws Exception {
+        LedgerHandle lh = bookkeeper.createLedger(3, 3, 2,
+                BookKeeper.DigestType.MAC, DICE_PASSWD);
+        try {
+            while (leader) {
+                Thread.sleep(1000);
+                int nextInt = r.nextInt(6) + 1;
+                lh.addEntry(Ints.toByteArray(nextInt));
+                System.out.println(&quot;Value = &quot; + nextInt
+                                   + &quot;, isLeader = &quot; + leader);
+            }
+        } finally {
+            lh.close();
+        }
+    }
+
+    void playDice() throws Exception {
+        while (true) {
+            if (leader) {
+                lead();
+            }
+        }
+    }
+</pre>
+
+
+
+<p>When we become the leader, we create a new ledger. A ledger is the basic unit of bookkeeper. It can be thought of as a segment of a larger log. At this moment we are only creating a single ledger, but later we will be creating multiple ledgers and connecting them together to create a shared log. For now, we just want to get data into a ledger.</p>
+
+<p>The ledger is created with a 3-3-2 configuration. These are the ensemble, the write quorum and the ack quorum. The ensemble is the number of bookies the data in the ledger will be stored on. All entries may not be stored on all bookies if the ensemble is larger than the write quorum. The write quorum is the number of bookies each entry is written to. The ack quorum is the number of bookies we must get a response from before we acknowledge the write to the client. In this case, there are 3 bookies, we write to all 3 every time, but we acknowledge to the client when we've received a response from 2. If the ensemble is larger than the write quorum, then entries will be striped across the bookies.</p>
+
+<p>The digest type and password are used for checksumming. They prevent clients from overwriting each others data in a misconfigured system. They're actually unnecessary in this example, but the client api requires them.</p>
+
+<p>Once the ledger is created we can write to it. <mark>addEntry()</mark> will append an entry onto the end of the ledger. Entries are byte arrays, so we convert the randomly generated integer into a byte array, using <a href="https://code.google.com/p/guava-libraries/">Guava</a>'s Ints utility, before adding it to the ledger.</p>
+
+<p>Once we are finished with a ledger we must close it. This is actually an important step and it fixes the content of the ledger. From this point on the ledger is immutable. It cannot be reopened for writing and its contents cannot be modified.</p>
+
+<p>Of course, we don't save a reference to the ledger anywhere, so once we have written it, no one else can ever access it, even to read it. This is what we will deal with in the next section.</p>
+
+<h1>Making the log available to others</h1>
+
+<p><a href="https://github.com/ivankelly/bookkeeper-tutorial/tree/sharing">(full code)</a></p>
+
+<p>Previously we have written to a single ledger. However, we have not provided a way to share this between instances. What's more, as a ledger is immutable, each leader will have to create its own ledger. So ultimately, when the application has run for a while, having changed leaders multiple times, we will end up with a list of ledgers. This list of ledgers represents the log of the application. Any new instance can print the same output as any preexisting instance by simply reading this log.</p>
+
+<p>This list of logs needs to be shared among all instances of the application. For this we will use zookeeper. </p>
+
+
+
+<pre class="prettyprint">
+public class Dice extends LeaderSelectorListenerAdapter implements Closeable {
+    ...
+
+    final static String DICE_LOG = &quot;/dice-log&quot;;
+</pre>
+
+
+
+<p>We define the path of the zookeeper znode in which we want to store the log. A znode in zookeeper is like a file. You can write and read byte arrays from a znode. However, the contents of a znode must be written and read as a whole, so it's best to only store small pieces of data there. Each time a znode is updated, a new version is assigned. This can be used for check-and-set operations, which is important to avoid race conditions in distributed systems.</p>
+
+
+
+<pre class="prettyprint">
+    void lead() throws Exception {
+        Stat stat = new Stat();
+        List&lt;Long&gt; ledgers;
+        boolean mustCreate = false;
+        try {
+            byte[] ledgerListBytes = curator.getData()
+                .storingStatIn(stat).forPath(DICE_LOG);
+            ledgers = listFromBytes(ledgerListBytes);
+        } catch (KeeperException.NoNodeException nne) {
+            ledgers = new ArrayList&lt;Long&gt;();
+            mustCreate = true;
+        }
+        for (Long previous : ledgers) {
+            LedgerHandle lh;
+            try {
+                lh = bookkeeper.openLedger(previous,
+                        BookKeeper.DigestType.MAC, DICE_PASSWD);
+            } catch (BKException.BKLedgerRecoveryException e) {
+                return;
+            }
+            Enumeration&lt;LedgerEntry&gt; entries
+                = lh.readEntries(0, lh.getLastAddConfirmed());
+
+            while (entries.hasMoreElements()) {
+                byte[] entryData = entries.nextElement().getEntry();
+                System.out.println(&quot;Value = &quot; + Ints.fromByteArray(entryData)
+                                   + &quot;, epoch = &quot; + lh.getId()
+                                   + &quot;, catchup&quot;);
+            }
+        }
+</pre>
+
+
+
+<p>We read the list of ledgers from <span class="caps">DICE</span>_LOG and store the version in stat. As the list of ledgers is in byte form, we need to convert into a java list. If this is the first time running, there will be no list of ledgers, and therefore no znode containing them. In this case a <mark>NoNodeException</mark> will occur. We take note of this using <mark>mustCreate</mark>, as it affects how will will update the list later.</p>
+
+<p>Once we have the list, we loop through them, opening the ledgers and printing their contents. It's important to note that the default open operation in bookkeeper is a fencing open. In a fencing open, anyone who is writing to the ledger will receive an exception when they try to write again. This is how we exclude other leaders.</p>
+
+
+
+<pre class="prettyprint">
+    void lead() throws Exception {
+        ...
+
+        LedgerHandle lh = bookkeeper.createLedger(3, 3, 2,
+                BookKeeper.DigestType.MAC, DICE_PASSWD);
+        ledgers.add(lh.getId());
+        byte[] ledgerListBytes = listToBytes(ledgers);
+        if (mustCreate) {
+            try {
+                curator.create().forPath(DICE_LOG, ledgerListBytes);
+            } catch (KeeperException.NodeExistsException nne) {
+                return;
+            }
+        } else {
+            try {
+                curator.setData()
+                    .withVersion(stat.getVersion())
+                    .forPath(DICE_LOG, ledgerListBytes);
+            } catch (KeeperException.BadVersionException bve) {
+                return;
+            }
+        }
+
+        try {
+            while (leader) {
+                Thread.sleep(1000);
+                int nextInt = r.nextInt(6) + 1;
+                lh.addEntry(Ints.toByteArray(nextInt));
+                System.out.println(&quot;Value = &quot; + nextInt
+                                   + &quot;, epoch = &quot; + lh.getId()
+                                   + &quot;, leading&quot;);
+            }
+            lh.close();
+        } catch (BKException e) {
+            return;
+        }
+    }
+</pre>
+
+
+
+<p>Once we have read all the previous ledgers, we create a new one and add it to the list. We must make sure this list is updated before writing to the ledger to avoid losing data. If <mark>create()</mark> or <mark>setData()</mark> throw an exception, it means that someone is trying to update the list concurrently. We must examine if we are still leader, and try again if we are. The retry is handled by the loop in <mark>playDice()</mark>.</p>
+
+<p>We can then write to the ledger as before. However, now we have to take care to handle the <mark><span class="caps">BKE</span>xception</mark>. If we receive an exception, it may mean that someone has fenced the ledger we are writing to. This means that someone else has opened it using <mark>openLedger()</mark>, so they must think that they are the leader. Like in the case of concurrent modifications to the ledger list, we must examine if we are still leader and then try again if so.</p>
+
+<p>Run a couple of instances of this on your machine. You'll see that when the leader changes, it will print out the history of what was written by previous leaders.</p>
+
+<p>However, we have a bug! When an instance becomes leader, it will print out the whole history, even if it has been leader before. So it is necessary to keep track of which updates we have seen been changes of leadership.</p>
+
+<h1>Tracking the updates</h1>
+
+<p><a href="https://github.com/ivankelly/bookkeeper-tutorial/tree/tracking">(full code)</a></p>
+
+<p>Tracking the updates is fairly simple. We just need to keep a record of the last thing we printed, and skip past it any time we become leader. </p>
+
+
+
+<pre class="prettyprint">
+    EntryId lead(EntryId skipPast) throws Exception {
+        EntryId lastDisplayedEntry = skipPast;
+</pre>
+
+
+
+<p>The signature for <mark>lead()</mark> needs to change so that the last displayed update is passed between different invocations. <mark>EntryId</mark> is a simple data structure, inside which we can store the ledger id and the entry id of the last update we have displayed.</p>
+
+
+
+<pre class="prettyprint">
+    EntryId lead(EntryId skipPast) throws Exception {
+        ...
+        List&lt;Long&gt; toRead = ledgers;
+        if (skipPast.getLedgerId() != -1) {
+            toRead = ledgers.subList(ledgers.indexOf(skipPast.getLedgerId()),
+                                     ledgers.size());
+        }
+
+        long nextEntry = skipPast.getEntryId() + 1;
+        for (Long previous : toRead) {
+            LedgerHandle lh;
+            try {
+                lh = bookkeeper.openLedger(previous,
+                        BookKeeper.DigestType.MAC, DICE_PASSWD);
+            } catch (BKException.BKLedgerRecoveryException e) {
+                return lastDisplayedEntry;
+            }
+
+            if (nextEntry &gt; lh.getLastAddConfirmed()) {
+                nextEntry = 0;
+                continue;
+            }
+            Enumeration&lt;LedgerEntry&gt; entries
+                = lh.readEntries(nextEntry, lh.getLastAddConfirmed());
+
+            while (entries.hasMoreElements()) {
+                LedgerEntry e = entries.nextElement();
+                byte[] entryData = e.getEntry();
+                System.out.println(&quot;Value = &quot; + Ints.fromByteArray(entryData)
+                                   + &quot;, epoch = &quot; + lh.getId()
+                                   + &quot;, catchup&quot;);
+                lastDisplayedEntry = new EntryId(lh.getId(), e.getEntryId());
+            }
+        }
+        ...
+</pre>
+
+
+
+<p>The algorithm for reading also changes. Instead of iterating through all the ledgers in the list we only iterate through any ledger which is greater to or equal to the ledger of the last displayed entry. We also skip past the entry id of the last displayed entry when calling <mark>readEntries()</mark>. The only special case we need to handle is if the last displayed entry is the last entry of a ledger. In this case, we set <mark>nextEntry</mark> to zero, and skip to the next ledger.</p>
+
+<p>Any time we do read an entry and display it, we update the last displayed entry to reflect this.</p>
+
+
+
+<pre class="prettyprint">
+    EntryId lead(EntryId skipPast) throws Exception {
+        ...
+
+        try {
+            while (leader) {
+                Thread.sleep(1000);
+                int nextInt = r.nextInt(6) + 1;
+                long entryId = lh.addEntry(Ints.toByteArray(nextInt));
+                System.out.println(&quot;Value = &quot; + nextInt
+                                   + &quot;, epoch = &quot; + lh.getId()
+                                   + &quot;, leading&quot;);
+                lastDisplayedEntry = new EntryId(lh.getId(), entryId);
+            }
+            lh.close();
+        } catch (BKException e) {
+            // let it fall through to the return
+        }
+        return lastDisplayedEntry;
+    }
+</pre>
+
+
+
+<p>Finally, we also update the last displayed entry any time we add a new entry to the log. With this change, new leaders will only print numbers which they haven't seen before. You can test this for yourself. Run two instances of the application. Stop the leader with Control-Z, and once the other instance has become leader, resume the first one (<mark>fg</mark>). Then kill the second leader. When the first leader becomes leader again, it will only print the number which it missed.</p>
+
+<h1>Tailing the log</h1>
+
+<p><a href="https://github.com/ivankelly/bookkeeper-tutorial/tree/tailing">(full code)</a></p>
+
+<p>Of course, it would be nicer if the followers could keep up to date with the leader in the background without having to wait to become leaders themselves. To do this we need to tail the log. For the most part this is very similar to how we read the previous ledgers when we become leader. However, how we open the ledgers is different. When we open the ledgers as leader, we need to ensure that no other instance can write to the ledgers from that point onwards. Therefore, we use a fencing open, which is the default <mark>openLedger()</mark> call in Bookkeeper. However, for tailing the log, we don't want to stop the leader from writing new updates, so we use a non-fenching open, which is the <mark>openLedgerNoRecovery()</mark> call in Bookkeeper.</p>
+
+<p>First we must modify <mark>playDice()</mark> to go into a following state when we're not the leader.</p>
+
+
+<pre class="prettyprint">
+    void playDice() throws Exception {
+        EntryId lastDisplayedEntry = new EntryId(-1, -1);
+        while (true) {
+            if (leader) {
+                lastDisplayedEntry = lead(lastDisplayedEntry);
+            } else {
+                lastDisplayedEntry = follow(lastDisplayedEntry);
+            }
+        }
+    }
+
+    EntryId follow(EntryId skipPast) throws Exception {
+        List&lt;Long&gt; ledgers = null;
+        while (ledgers == null) {
+            try {
+                byte[] ledgerListBytes = curator.getData()
+                    .forPath(DICE_LOG);
+                ledgers = listFromBytes(ledgerListBytes);
+                if (skipPast.getLedgerId() != -1) {
+                    ledgers = ledgers.subList(ledgers.indexOf(skipPast.getLedgerId()),
+                                              ledgers.size());
+                }
+            } catch (KeeperException.NoNodeException nne) {
+                Thread.sleep(1000);
+            }
+        }
+</pre>
+
+
+
+<p>The first part of following is almost identical to leading. We read the list of ledgers from zookeeper and trim the list to only include ledgers which we have displayed already. A thing to note here, is that if we go into following mode during the first run of the application, and the leader hasn't created the list of ledgers in zookeeper yet we will get an exception. If this occurs we try again after 1 second.</p>
+
+<p>Once we have the list, we go into the main tailing loop.</p>
+
+
+
+<pre class="prettyprint">
+    EntryId follow(EntryId skipPast) throws Exception {
+        ...
+
+        EntryId lastReadEntry = skipPast;
+        while (!leader) {
+            for (long previous : ledgers) {
+                ...
+            }
+            byte[] ledgerListBytes = curator.getData()
+                .forPath(DICE_LOG);
+            ledgers = listFromBytes(ledgerListBytes);
+            ledgers = ledgers.subList(ledgers.indexOf(lastReadEntry.getLedgerId())+1,
+                                      ledgers.size());
+        }
+        return lastReadEntry;
+    }
+</pre>
+
+
+
+<p>While we are still leader, we loop over all ledgers in the ledgers list, printing their content. Once we have finished with the current list of ledgers, we check zookeeper to see if any new ledgers have been added to the list. This looks like it would run in a tight loop, but that is not the case. Ledger reading loop will wait until the last ledger in the list is closed before exiting the loop. When the last ledger in the list is closed, it means that the leader must have changed, so there must be a new ledger in the list to read.</p>
+
+
+
+<pre class="prettyprint">
+            for (long previous : ledgers) {
+                boolean isClosed = false;
+                long nextEntry = 0;
+                while (!isClosed &amp;&amp; !leader) {
+                    if (lastReadEntry.getLedgerId() == previous) {
+                        nextEntry = lastReadEntry.getEntryId() + 1;
+                    }
+                    isClosed = bookkeeper.isClosed(previous);
+                    LedgerHandle lh = bookkeeper.openLedgerNoRecovery(previous,
+                            BookKeeper.DigestType.MAC, DICE_PASSWD);
+
+                    if (nextEntry &lt;= lh.getLastAddConfirmed()) {
+                        ... // read all entries from nextEntry to last add confirmed
+                    }
+                    if (isClosed) {
+                        break;
+                    }
+                    Thread.sleep(1000);
+                } 
+            }
+</pre>
+
+
+
+<p>For each ledger we enter into an inner loop. First we check if the ledger has been closed. If so, once we have read all the entries that we can, we need to reopen the ledger to check for any new entries. We continue like this until the ledger is either closed, or we become leader.</p>
+
+<p>Note that we are using <mark>openLedgerNoRecovery()</mark> here. The value returned by last add confirmed will change after each opening if there are new entries which can be read. The last add confirmed is a variable maintained by the leader. It is the last entry written for which it has received an <span class="caps">ACK </span>quorum of acknowledgements. In our case, this means that the entry has been acknowledged on at least 2 bookies. It also guarantees that each entry before it in that ledger has been acknowledged on 2 bookies.</p>
+
+<p>Once we have read all entries, we check isClosed to see if we need to check this ledger again. If not, we break out of the loop and move onto the next ledger. Otherwise, we wait a second and try again.</p>
+
+<h1>Wrap up</h1>
+
+<p>Now you have a fully distributed dice application. Not very useful, but it should give you some idea of what is required to make an application fault tolerant without losing consistency. Play around with the application. Run many instances. Kill a few leaders. You will always see the same sequence of number printed to the screen. If not, then you have found a bug, please let us know.</p>
+
+<h1>What's next?</h1>
+
+<p>The dice application we've written is just and example and is pretty useless in the real world. But the principles contained therein could be used to replicate pretty much any service. Imagine a simple key value store. This could be made replicated by adding all create, put and delete operations to a replicated log. Multiple logs could be used if you want to shard your store across many servers. And there are many possibilities.</p>
+
+<p>However, this tutorial doesn't address some issues that would be important in a real implementation. For starters, the log of the dice application will keep growing forever, eventually filling up all your disks and grinding you to a halt. Avoiding this problem depends on your individual usecase. For example, if you have a key value store, you can take a snapshot of the store every so often, and then trim the start of the log to remove anything that had been applied by the time the snapshot was taken. Trimming simply means removing ledgers from the start of the ledger list. For a messaging application, you could keep a record of what each subscriber has consumed and then trim the log based on that.</p>
+
+<p>Note that the tutorial application only uses synchronous <span class="caps">API</span>s. The bookkeeper client does also have asynchronous <span class="caps">API</span>s, which allow for higher throughput when writing. However, this means that you have to manage your state more carefully.</p>
+
+<script src="https://google-code-prettify.googlecode.com/svn/loader/run_prettify.js"></script>
+
+    </div>
+    <footer class="footer">
+      <div class="container">
+        <p class="text-muted">Copyright &copy; 2014 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.<br/>
+	Apache BookKeeper, BookKeeper, Apache, Apache ZooKeeper, ZooKeeper, the Apache feather logo, and the Apache BookKeeper project logo are trademarks of The Apache Software Foundation.</p>
+      </div>
+    </footer>
+
+    <script src="http://code.jquery.com/jquery.js"></script>
+    <script src="/js/bootstrap.min.js"></script>
+  </body>
+</html>

Added: websites/staging/bookkeeper/trunk/content/docs/r4.4.0/doc.html
==============================================================================
--- websites/staging/bookkeeper/trunk/content/docs/r4.4.0/doc.html (added)
+++ websites/staging/bookkeeper/trunk/content/docs/r4.4.0/doc.html Sun May 15 21:40:24 2016
@@ -0,0 +1,141 @@
+<!DOCTYPE html>
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE- 2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<html>
+  <head>
+    <title>Apache BookKeeper - </title>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <!-- Bootstrap -->
+    <link href="/css/bootstrap.min.css" rel="stylesheet">
+    <link href="/css/bootstrap-responsive.min.css" rel="stylesheet">
+    <link href="/css/styles.css" rel="stylesheet">
+  </head>
+  <body>
+    <header class="navbar navbar-inverse navbar-static-top" role="banner">
+      <div class="container">
+    	<div class="navbar-header hidden-xs hidden-sm">
+    	  <a class="navbar-brand navbar-logo" href="/"><img class="img-responsive" src="/img/bookkeeper_blk40.png" alt="Bookkeeper Logo" /></a>
+    	</div>
+    	<div class="navbar-header">
+    	  <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
+            <span class="sr-only">Toggle navigation</span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+    	  </button>
+    	  <a class="navbar-brand" href="/">Apache BookKeeper</a>
+    	</div>
+    	<nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
+    	  <ul class="nav navbar-nav">
+    	    <li><a href="/releases.html">Download</a></li>
+
+    	    <li class="dropdown">
+    	      <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Documentation<span class="caret"></span></a>
+    	      <ul class="dropdown-menu" role="menu">
+		<li><a href="/docs/master">Latest (master)</a></li>
+		<li><ul>
+		    <li><a href="/docs/master/apidocs">Java API docs</a></li>
+		    <li><a href="/docs/master/bookkeeperTutorial.html">Tutorial</a></li>
+		    <li><a href="/docs/master/bookkeeperConfig.html">Admin guide</a></li>
+		</ul><li>
+                <li><a href="/docs/r4.4.0">Release 4.4.0</a></li>
+                <li class="divider"></li>
+                <li>Older releases</li>
+                <li><a href="/docs/r4.3.2">Release 4.3.2</a></li>
+                <li><a href="/docs/r4.3.1">Release 4.3.1</a></li>
+                <li><a href="/docs/r4.3.0">Release 4.3.0</a></li>
+                <li><a href="/docs/r4.2.4">Release 4.2.4</a></li>
+                <li><a href="/docs/r4.2.3">Release 4.2.3</a></li>
+                <li><a href="/docs/r4.2.2">Release 4.2.2</a></li>
+                <li><a href="/docs/r4.2.1">Release 4.2.1</a></li>
+                <li><a href="/docs/r4.2.0">Release 4.2.0</a></li>
+                <li><a href="/docs/r4.1.0">Release 4.1.0</a></li>
+                <li><a href="/docs/r4.0.0">Release 4.0.0</a></li>
+              </ul>
+            </li>
+            
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Get Involved<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/lists.html">Mailing Lists</a></li>
+                <li><a href="/irc.html">IRC</a></li>
+                <li><a href="/svn.html">Version Control</a></li>
+                <li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+              </ul>
+            </li>
+
+            <li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+            <!--<li><a href="#">Hedwig</a></li>//-->
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project Info<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/credits.html">Who are we?</a></li>
+                <li><a href="/bylaws.html">Bylaws</a></li>
+                <li><a href="http://www.apache.org/licenses/">License</a></li>
+                <li class="divider"></li>
+                <li><a href="/privacy.html">Privacy Policy</a></li>
+                <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsership</a></li>
+                <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+              </ul>
+            </li>
+          </ul>
+          <script>
+            (function() {
+            var cx = '017580107654524087317:iqnsyimpydg';
+            var gcse = document.createElement('script');
+            gcse.type = 'text/javascript';
+            gcse.async = true;
+            gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+            '//www.google.com/cse/cse.js?cx=' + cx;
+            var s = document.getElementsByTagName('script')[0];
+            s.parentNode.insertBefore(gcse, s);
+            })();
+          </script>
+          
+          <div class="navbar-form navbar-right visible-lg" id="googlebox">
+            <gcse:searchbox-only></gcse:searchbox-only>
+          </div>
+        </nav>
+      </div>
+    </header>
+    <div class="container">
+
+ <p>In the documentation directory, you'll find:</p>
+
+<ul>
+<li><code>build.txt</code>: Building Hedwig, or how to set up Hedwig</li>
+<li><code>user.txt</code>: User's Guide, or how to program against the Hedwig <span class="caps">API </span>and how to run it</li>
+<li><code>dev.txt</code>: Developer's Guide, or Hedwig internals and hacking details</li>
+</ul>
+
+<p>These documents are all written in the <a href="http://johnmacfarlane.net/pandoc/">Pandoc</a> dialect of <a href="http://daringfireball.net/projects/markdown/">Markdown</a>. This makes them readable as plain text files, but also capable of generating <span class="caps">HTML </span>or LaTeX documentation.</p>
+
+<p>Documents are wrapped at 80 chars and use 2-space indentation.</p>
+
+    </div>
+    <footer class="footer">
+      <div class="container">
+        <p class="text-muted">Copyright &copy; 2014 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.<br/>
+	Apache BookKeeper, BookKeeper, Apache, Apache ZooKeeper, ZooKeeper, the Apache feather logo, and the Apache BookKeeper project logo are trademarks of The Apache Software Foundation.</p>
+      </div>
+    </footer>
+
+    <script src="http://code.jquery.com/jquery.js"></script>
+    <script src="/js/bootstrap.min.js"></script>
+  </body>
+</html>

Added: websites/staging/bookkeeper/trunk/content/docs/r4.4.0/index.html
==============================================================================
--- websites/staging/bookkeeper/trunk/content/docs/r4.4.0/index.html (added)
+++ websites/staging/bookkeeper/trunk/content/docs/r4.4.0/index.html Sun May 15 21:40:24 2016
@@ -0,0 +1,176 @@
+<!DOCTYPE html>
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE- 2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<html>
+  <head>
+    <title>Apache BookKeeper - BookKeeper Documentation</title>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <!-- Bootstrap -->
+    <link href="/css/bootstrap.min.css" rel="stylesheet">
+    <link href="/css/bootstrap-responsive.min.css" rel="stylesheet">
+    <link href="/css/styles.css" rel="stylesheet">
+  </head>
+  <body>
+    <header class="navbar navbar-inverse navbar-static-top" role="banner">
+      <div class="container">
+    	<div class="navbar-header hidden-xs hidden-sm">
+    	  <a class="navbar-brand navbar-logo" href="/"><img class="img-responsive" src="/img/bookkeeper_blk40.png" alt="Bookkeeper Logo" /></a>
+    	</div>
+    	<div class="navbar-header">
+    	  <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
+            <span class="sr-only">Toggle navigation</span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+    	  </button>
+    	  <a class="navbar-brand" href="/">Apache BookKeeper</a>
+    	</div>
+    	<nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
+    	  <ul class="nav navbar-nav">
+    	    <li><a href="/releases.html">Download</a></li>
+
+    	    <li class="dropdown">
+    	      <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Documentation<span class="caret"></span></a>
+    	      <ul class="dropdown-menu" role="menu">
+		<li><a href="/docs/master">Latest (master)</a></li>
+		<li><ul>
+		    <li><a href="/docs/master/apidocs">Java API docs</a></li>
+		    <li><a href="/docs/master/bookkeeperTutorial.html">Tutorial</a></li>
+		    <li><a href="/docs/master/bookkeeperConfig.html">Admin guide</a></li>
+		</ul><li>
+                <li><a href="/docs/r4.4.0">Release 4.4.0</a></li>
+                <li class="divider"></li>
+                <li>Older releases</li>
+                <li><a href="/docs/r4.3.2">Release 4.3.2</a></li>
+                <li><a href="/docs/r4.3.1">Release 4.3.1</a></li>
+                <li><a href="/docs/r4.3.0">Release 4.3.0</a></li>
+                <li><a href="/docs/r4.2.4">Release 4.2.4</a></li>
+                <li><a href="/docs/r4.2.3">Release 4.2.3</a></li>
+                <li><a href="/docs/r4.2.2">Release 4.2.2</a></li>
+                <li><a href="/docs/r4.2.1">Release 4.2.1</a></li>
+                <li><a href="/docs/r4.2.0">Release 4.2.0</a></li>
+                <li><a href="/docs/r4.1.0">Release 4.1.0</a></li>
+                <li><a href="/docs/r4.0.0">Release 4.0.0</a></li>
+              </ul>
+            </li>
+            
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Get Involved<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/lists.html">Mailing Lists</a></li>
+                <li><a href="/irc.html">IRC</a></li>
+                <li><a href="/svn.html">Version Control</a></li>
+                <li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+              </ul>
+            </li>
+
+            <li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+            <!--<li><a href="#">Hedwig</a></li>//-->
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project Info<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/credits.html">Who are we?</a></li>
+                <li><a href="/bylaws.html">Bylaws</a></li>
+                <li><a href="http://www.apache.org/licenses/">License</a></li>
+                <li class="divider"></li>
+                <li><a href="/privacy.html">Privacy Policy</a></li>
+                <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsership</a></li>
+                <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+              </ul>
+            </li>
+          </ul>
+          <script>
+            (function() {
+            var cx = '017580107654524087317:iqnsyimpydg';
+            var gcse = document.createElement('script');
+            gcse.type = 'text/javascript';
+            gcse.async = true;
+            gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+            '//www.google.com/cse/cse.js?cx=' + cx;
+            var s = document.getElementsByTagName('script')[0];
+            s.parentNode.insertBefore(gcse, s);
+            })();
+          </script>
+          
+          <div class="navbar-form navbar-right visible-lg" id="googlebox">
+            <gcse:searchbox-only></gcse:searchbox-only>
+          </div>
+        </nav>
+      </div>
+    </header>
+    <div class="container">
+
+ <p>How to start with BookKeeper depends on who you are...</p>
+
+<p><strong>Developers</strong> who are new to BookKeeper should start with the <a href="./bookkeeperTutorial.html">BookKeeper Tutorial</a>. The tutorial shows you how to build a basic distributed system using BookKeeper. <a href="./bookkeeperLedgers2Logs.html">Turning ledgers into logs</a> gives a briefer description of the principles behind the logs used in the tutorial.</p>
+
+<p>Once familiar with the basic concepts, developers can consult the <a href="./apidocs">BookKeeper Java <span class="caps">API </span>documentation</a>.</p>
+
+<p><strong>Administrators</strong> will be more interested in the <a href="./bookkeeperConfig.html">BookKeeper Admin guide</a>. It describes the steps involved in setting up and maintaining a cluster. The available configuration parameters can be found <a href="./bookieConfigParams.html">here</a>. An important aspect of BookKeeper is how it deals with the failure of storage nodes. This is covered in <a href="./bookieRecovery.html">Bookie Recovery</a>.</p>
+
+<p><strong>Contributor</strong> documentation is less organized, <a href="./bookkeeperInternals.html">BookKeeper Internals</a> is a good place to start. From there you can check out our <a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">wiki</a> and ask questions on our <a href="/lists.html">mailing lists</a> or <a href="/irc.html"><span class="caps">IRC</span></a>.</p>
+
+<h3>All documents</h3>
+
+<ul>
+<li>Overview<ul>
+<li><a href="./bookkeeperStarted.html">Getting started</a></li>
+<li><a href="./bookkeeperOverview.html">Overview</a></li>
+</ul></li>
+</ul>
+
+<ul>
+<li>Developers<ul>
+<li><a href="./bookkeeperTutorial.html">BookKeeper Tutorial</a></li>
+<li><a href="./bookkeeperLedgers2Logs.html">Turning ledgers into logs</a></li>
+<li><a href="./apidocs">BookKeeper Java <span class="caps">API </span>documentation</a></li>
+<li><a href="./bookkeeperProgrammer.html" title="old">Programmer's Guide</a></li>
+<li><a href="./bookkeeperConfigParams.html" title="old">BookKeeper Configuration Parameters</a></li>
+</ul></li>
+</ul>
+
+
+<ul>
+<li>Administrators<ul>
+<li><a href="./bookkeeperConfig.html">Admin Guide</a></li>
+<li><a href="./bookkeeperJMX.html">BookKeeper <span class="caps">JMX</span></a></li>
+<li><a href="./bookieConfigParams.html" title="old">Bookie Server Configuration Parameters</a></li>
+<li><a href="./bookieRecovery.html">Bookie Recovery</a></li>
+</ul></li>
+</ul>
+
+<ul>
+<li>Contributors<ul>
+<li><a href="./bookkeeperInternals.html">BookKeeper Internals</a></li>
+<li><a href="./bookkeeperMetadata.html">BookKeeper Metadata Management</a></li>
+<li><a href="./metastore.textile">Metastore Interface</a></li>
+</ul></li>
+</ul>
+
+    </div>
+    <footer class="footer">
+      <div class="container">
+        <p class="text-muted">Copyright &copy; 2014 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.<br/>
+	Apache BookKeeper, BookKeeper, Apache, Apache ZooKeeper, ZooKeeper, the Apache feather logo, and the Apache BookKeeper project logo are trademarks of The Apache Software Foundation.</p>
+      </div>
+    </footer>
+
+    <script src="http://code.jquery.com/jquery.js"></script>
+    <script src="/js/bootstrap.min.js"></script>
+  </body>
+</html>

Added: websites/staging/bookkeeper/trunk/content/docs/r4.4.0/metastore.html
==============================================================================
--- websites/staging/bookkeeper/trunk/content/docs/r4.4.0/metastore.html (added)
+++ websites/staging/bookkeeper/trunk/content/docs/r4.4.0/metastore.html Sun May 15 21:40:24 2016
@@ -0,0 +1,169 @@
+<!DOCTYPE html>
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    (the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE- 2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+-->
+<html>
+  <head>
+    <title>Apache BookKeeper - Metastore Interface</title>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <!-- Bootstrap -->
+    <link href="/css/bootstrap.min.css" rel="stylesheet">
+    <link href="/css/bootstrap-responsive.min.css" rel="stylesheet">
+    <link href="/css/styles.css" rel="stylesheet">
+  </head>
+  <body>
+    <header class="navbar navbar-inverse navbar-static-top" role="banner">
+      <div class="container">
+    	<div class="navbar-header hidden-xs hidden-sm">
+    	  <a class="navbar-brand navbar-logo" href="/"><img class="img-responsive" src="/img/bookkeeper_blk40.png" alt="Bookkeeper Logo" /></a>
+    	</div>
+    	<div class="navbar-header">
+    	  <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse">
+            <span class="sr-only">Toggle navigation</span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+    	  </button>
+    	  <a class="navbar-brand" href="/">Apache BookKeeper</a>
+    	</div>
+    	<nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation">
+    	  <ul class="nav navbar-nav">
+    	    <li><a href="/releases.html">Download</a></li>
+
+    	    <li class="dropdown">
+    	      <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Documentation<span class="caret"></span></a>
+    	      <ul class="dropdown-menu" role="menu">
+		<li><a href="/docs/master">Latest (master)</a></li>
+		<li><ul>
+		    <li><a href="/docs/master/apidocs">Java API docs</a></li>
+		    <li><a href="/docs/master/bookkeeperTutorial.html">Tutorial</a></li>
+		    <li><a href="/docs/master/bookkeeperConfig.html">Admin guide</a></li>
+		</ul><li>
+                <li><a href="/docs/r4.4.0">Release 4.4.0</a></li>
+                <li class="divider"></li>
+                <li>Older releases</li>
+                <li><a href="/docs/r4.3.2">Release 4.3.2</a></li>
+                <li><a href="/docs/r4.3.1">Release 4.3.1</a></li>
+                <li><a href="/docs/r4.3.0">Release 4.3.0</a></li>
+                <li><a href="/docs/r4.2.4">Release 4.2.4</a></li>
+                <li><a href="/docs/r4.2.3">Release 4.2.3</a></li>
+                <li><a href="/docs/r4.2.2">Release 4.2.2</a></li>
+                <li><a href="/docs/r4.2.1">Release 4.2.1</a></li>
+                <li><a href="/docs/r4.2.0">Release 4.2.0</a></li>
+                <li><a href="/docs/r4.1.0">Release 4.1.0</a></li>
+                <li><a href="/docs/r4.0.0">Release 4.0.0</a></li>
+              </ul>
+            </li>
+            
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Get Involved<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/lists.html">Mailing Lists</a></li>
+                <li><a href="/irc.html">IRC</a></li>
+                <li><a href="/svn.html">Version Control</a></li>
+                <li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+              </ul>
+            </li>
+
+            <li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+            <!--<li><a href="#">Hedwig</a></li>//-->
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project Info<span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="/credits.html">Who are we?</a></li>
+                <li><a href="/bylaws.html">Bylaws</a></li>
+                <li><a href="http://www.apache.org/licenses/">License</a></li>
+                <li class="divider"></li>
+                <li><a href="/privacy.html">Privacy Policy</a></li>
+                <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsership</a></li>
+                <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+              </ul>
+            </li>
+          </ul>
+          <script>
+            (function() {
+            var cx = '017580107654524087317:iqnsyimpydg';
+            var gcse = document.createElement('script');
+            gcse.type = 'text/javascript';
+            gcse.async = true;
+            gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
+            '//www.google.com/cse/cse.js?cx=' + cx;
+            var s = document.getElementsByTagName('script')[0];
+            s.parentNode.insertBefore(gcse, s);
+            })();
+          </script>
+          
+          <div class="navbar-form navbar-right visible-lg" id="googlebox">
+            <gcse:searchbox-only></gcse:searchbox-only>
+          </div>
+        </nav>
+      </div>
+    </header>
+    <div class="container">
+
+ <h1>Metastore Interface</h1>
+
+<p>Although Apache BookKeeper provides <a href="./bookkeeperMetadata.html">LedgerManager</a> for users to plugin different metadata storages for BookKeeper, it is quite difficult to implement a correct and efficient manager version based on the knowledge. The <i>MetaStore</i> interface extracts the commonality of the metadata storage interfaces and is provided for users to focus on adapting the underlying storage itself w/o having to worry about the detailed logic for BookKeeper.</p>
+
+<h2>MetaStore</h2>
+
+<p>The <i>MetaStore</i> interface provide users with access to <i>MetastoreTable__s used for BookKeeper metadata management. There are two kinds of table defined in a <i>MetaStore</i>, <i>MetastoreTable</i> which provides basic <i><span class="caps">PUT</span></i>,__GET</i>,__REMOVE__,__SCAN__ operations and which does not assume any ordering requirements from the underlying storage; and <i>MetastoreScannableTable</i> which is derived from <i>MetastoreTable</i>, but <strong>does</strong> assume that data is stored in key order in the underlying storage.</p>
+
+<ul>
+<li><code>getName</code>: Return the name of the <i>MetaStore</i>.</li>
+<li><code>getVersion</code>: Return current <i>MetaStore</i> plugin version.</li>
+<li><code>init</code>: Initialize the <i>MetaStore</i> library with the given configuration and its version.</li>
+<li><code>close</code>: Close the <i>MetaStore</i>, freeing all resources. i.e. release all the open connections and occupied memory etc.</li>
+<li><code>createTable</code>: Create a table instance to access the data stored in it. A table name is given to locate the table. An <i>MetastoreTable</i> object is returned.</li>
+<li><code>createScannableTable</code>: Similar as <i>createTable</i>, but returns <i>MetastoreScannableTable</i> rather then <i>MetastoreTable</i> object. If the underlying table is not an ordered table, <i>MetastoreException</i> should be thrown.</li>
+</ul>
+
+<h2>MetaStore Table</h2>
+
+<p><i>MetastoreTable</i> is a basic unit in a <i>MetaStore</i>, which is used to handle different types of metadata, i.e. A <i>MetastoreTable</i> is used to store metadata for ledgers, while the other <i>MetastoreTable</i> is used to store metadata for topic persistence info. The interface for a <i>MetastoreTable</i> is quite simple:</p>
+
+<ul>
+<li><code>get</code>: Retrieve a entry by a given <i>key</i>. <i>OK</i> and its current version in metadata storage is returned when succeed. <i>NoKey</i> returned for a non-existent key. If <i>fields</i> are specified, return only the specified fields for the key.</li>
+<li><code>put</code>: Put the given <i>value</i> associated with <i>key</i> with given <i>version</i>. The value is only updated when the given <i>version</i> equals the current version in metadata storage. A new <i>version</i> should be returned when updated successfully. <i>NoKey</i> is returned for a non-existent key, <i>BadVersion</i> is returned when an update is attempted with a <i>version</i> which does not match the one in the metadata store.</li>
+<li><code>remove</code>: Remove the given <i>value</i> associated with <i>key</i>. The value is only removed when the given <i>version</i> equals its current version in metadata storage. <i>NoKey</i> is returned for a non-existent key, <i>BadVersion</i> is returned when remove is attempted with a <i>version</i> which does not match.</li>
+<li><code>openCursor</code>: Open a <i>cursor</i> to iterate over all the entries of a table. The returned cursor doesn't need to guarantee any order and transaction.</li>
+</ul>
+
+<h2>MetaStore Scannable Table</h2>
+
+<p><i>MetastoreScannableTable</i> is identical to a <i>MetastoreTable</i> except that it provides an addition interface to iterate over entries in the table in key order.</p>
+
+<ul>
+<li><code>openCursor</code>: Open a <i>cursor</i> to iterate over all the entries of a table between the key range of <i>firstKey</i> and <i>lastKey</i>.</li>
+</ul>
+
+<h2>How to organize your metadata.</h2>
+
+<p>Some metadata in BookKeeper does not need to be stored in the order of the ledger id or the topic. You could use kind of hash table to store metadata for them. These metadata are topic ownership and topic persistence info. Besides that, subscription state and ledger metadata must be stored in key order due to the current logic in BookKeeper.</p>
+
+    </div>
+    <footer class="footer">
+      <div class="container">
+        <p class="text-muted">Copyright &copy; 2014 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.<br/>
+	Apache BookKeeper, BookKeeper, Apache, Apache ZooKeeper, ZooKeeper, the Apache feather logo, and the Apache BookKeeper project logo are trademarks of The Apache Software Foundation.</p>
+      </div>
+    </footer>
+
+    <script src="http://code.jquery.com/jquery.js"></script>
+    <script src="/js/bootstrap.min.js"></script>
+  </body>
+</html>



Mime
View raw message