Return-Path: X-Original-To: apmail-zookeeper-commits-archive@www.apache.org Delivered-To: apmail-zookeeper-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 81607827E for ; Mon, 8 Aug 2011 23:28:34 +0000 (UTC) Received: (qmail 13857 invoked by uid 500); 8 Aug 2011 23:28:34 -0000 Delivered-To: apmail-zookeeper-commits-archive@zookeeper.apache.org Received: (qmail 13821 invoked by uid 500); 8 Aug 2011 23:28:34 -0000 Mailing-List: contact commits-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ Delivered-To: mailing list commits@zookeeper.apache.org Received: (qmail 13739 invoked by uid 99); 8 Aug 2011 23:28:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Aug 2011 23:28:33 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Aug 2011 23:28:29 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 1F5FA238885D for ; Mon, 8 Aug 2011 23:28:10 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r794047 [1/2] - in /websites/production/zookeeper: ./ content/bookkeeper/ content/bookkeeper/docs/trunk/ content/doc/trunk/ content/doc/trunk/skin/ Date: Mon, 08 Aug 2011 23:28:07 -0000 To: commits@zookeeper.apache.org From: breed@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20110808232810.1F5FA238885D@eris.apache.org> Author: breed Date: Mon Aug 8 23:28:06 2011 New Revision: 794047 Log: Publishing merge to zookeeper site by breed Modified: websites/production/zookeeper/ (props changed) websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html websites/production/zookeeper/content/bookkeeper/index.html websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.html websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.pdf websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.html websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.pdf websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.html websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.pdf websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.html websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.pdf websites/production/zookeeper/content/doc/trunk/bookkeeperStream.html websites/production/zookeeper/content/doc/trunk/bookkeeperStream.pdf websites/production/zookeeper/content/doc/trunk/index.html websites/production/zookeeper/content/doc/trunk/index.pdf websites/production/zookeeper/content/doc/trunk/javaExample.html websites/production/zookeeper/content/doc/trunk/javaExample.pdf websites/production/zookeeper/content/doc/trunk/linkmap.html websites/production/zookeeper/content/doc/trunk/linkmap.pdf websites/production/zookeeper/content/doc/trunk/recipes.html websites/production/zookeeper/content/doc/trunk/recipes.pdf websites/production/zookeeper/content/doc/trunk/releasenotes.html websites/production/zookeeper/content/doc/trunk/releasenotes.pdf websites/production/zookeeper/content/doc/trunk/skin/basic.css websites/production/zookeeper/content/doc/trunk/skin/screen.css websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.html websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.pdf websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.html websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.pdf websites/production/zookeeper/content/doc/trunk/zookeeperInternals.html websites/production/zookeeper/content/doc/trunk/zookeeperInternals.pdf websites/production/zookeeper/content/doc/trunk/zookeeperJMX.html websites/production/zookeeper/content/doc/trunk/zookeeperJMX.pdf websites/production/zookeeper/content/doc/trunk/zookeeperObservers.html websites/production/zookeeper/content/doc/trunk/zookeeperObservers.pdf websites/production/zookeeper/content/doc/trunk/zookeeperOver.html websites/production/zookeeper/content/doc/trunk/zookeeperOver.pdf websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.html websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.pdf websites/production/zookeeper/content/doc/trunk/zookeeperQuotas.html websites/production/zookeeper/content/doc/trunk/zookeeperQuotas.pdf websites/production/zookeeper/content/doc/trunk/zookeeperStarted.html websites/production/zookeeper/content/doc/trunk/zookeeperStarted.pdf websites/production/zookeeper/content/doc/trunk/zookeeperTutorial.html websites/production/zookeeper/content/doc/trunk/zookeeperTutorial.pdf Propchange: websites/production/zookeeper/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Mon Aug 8 23:28:06 2011 @@ -1 +1 @@ -/websites/staging/zookeeper/trunk:779660-791715 +/websites/staging/zookeeper/trunk:779660-794046 Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html ============================================================================== --- websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html (original) +++ websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html Mon Aug 8 23:28:06 2011 @@ -205,7 +205,7 @@

public LedgerHandle openLedger(long lId, DigestType type, byte passwd[]) throws InterruptedException, BKException

    -
  • ledgerId is the ledger identifier;
    • +
  • lId is the ledger identifier;
  • type is the type of digest used with entries: either MAC or CRC32.
  • passwd is a password to access the ledger (used only in the case of VERIFIABLE ledgers);
@@ -227,6 +227,34 @@
  • ctx is control object used for accountability purposes.
    • +

    The two calls above to open a ledger recover the ledger if it has not been closed properly before it reads entries from it. It is also possible to open a ledger and read from it without triggering the recovery process with the following methods of org.apache.bookkeeper.client.BookKeeper .

    + +

    Synchronous open:

    + +

    public LedgerHandle openLedgerNoRecovery(long lId, DigestType type, byte passwd[]) throws InterruptedException, BKException

    + +
      +
    • lId is the ledger identifier;
      • +
    • type is the type of digest used with entries: either MAC or CRC32.
      • +
    • passwd is a password to access the ledger (used only in the case of VERIFIABLE ledgers);
      • +
    + + +

    Asynchronous open:

    + +

    public void asyncOpenLedgerNoRecovery(long lId, DigestType type, byte passwd[], OpenCallback cb, Object ctx)

    + +

    It also takes a a ledger identifier and a password. Additionaly, it takes a callback object cb and a control object ctx . The callback object must implement the OpenCallback interface in org.apache.bookkeeper.client.AsyncCallback , and a class implementing it has to implement a method called openComplete that has the following signature:

    + +

    public void openComplete(int rc, LedgerHandle lh, Object ctx)

    + +

    where:

    + +
      +
    • rc is a return code (please refer to org.apache.bookeeper.client.BKDefs for a list);
      • +
    • lh is a LedgerHandle object to manipulate a ledger;
      • +
    • ctx is control object used for accountability purposes.
      • +

    Reading from ledger

    @@ -287,6 +315,30 @@
  • rc is a return code (please refer to org.apache.bookeeper.client.BKDefs for a list);
  • ctx is control object used for accountability purposes.
    • + +

    Reading the last confirmed entry from a ledger

    + +

    When reading from an open ledger (opening without recovery enables it), it is often necessary to read the last confirmed hint that the writer client writes along with every entry. The general idea is to obtain the hint from a quorum and return the largest value. The following methods belong to org.apache.bookkeeper.client.BookKeeper .

    + +

    Synchronous read:

    + +

    public long readLastConfirmed()

    + +

    Asynchronous read:

    + +

    public void asyncReadLastConfirmed(ReadLastConfirmedComplete cb, Object ctx)
    +
    +p. It takes a callback object cb and a control object ctx . The callback object must implement the ReadLastConfirmedCallback interface in org.apache.bookkeeper.client.AsyncCallback , and a class implementing it has to implement a method called readLastConfirmedComplete that has the following signature:

    + +

    public void readLastConfirmedComplete (int rc, long lastConfirmed, Object ctx)

    + +

    where:

    + +
      +
    • rc is a return code (please refer to org.apache.bookeeper.client.BKDefs for a list);
      • +
    • lastConfirmed is the maximum last confirmed hint received among a quorum of bookies;
      • +
    • ctx is control object used for accountability purposes.
      • +
    Modified: websites/production/zookeeper/content/bookkeeper/index.html ============================================================================== --- websites/production/zookeeper/content/bookkeeper/index.html (original) +++ websites/production/zookeeper/content/bookkeeper/index.html Mon Aug 8 23:28:06 2011 @@ -55,8 +55,6 @@

    Welcome to Apache BookKeeper™

    -

    this site is still under construction

    -

    The Apache BookKeeper subproject of ZooKeeper is made up of a distributed logging service called BookKeeper and a distribute publish/subscribe system build on top of BookKeeper called Hedwig.

    What is the BookKeeper?

    Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.html (original) +++ websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + BookKeeper Administrator's Guide @@ -204,6 +204,7 @@ document.write("Last Published: " + docu

    BookKeeper Administrator's Guide

    Setup Guide

    +
    • @@ -222,6 +223,7 @@ document.write("Last Published: " + docu
    +
    @@ -229,7 +231,7 @@ document.write("Last Published: " + docu - +

    Deployment

    This section contains information about deploying BookKeeper and @@ -267,7 +269,7 @@ document.write("Last Published: " + docu (BookKeeper storage servers). The third section explains how we use ZooKeeper and our requirements with respect to ZooKeeper.

    - +

    System requirements

    A typical BookKeeper installation comprises a set of bookies and a set of ZooKeeper replicas. The exact number of bookies depends on the quorum mode, desired throughput, and number of clients using this installation simultaneously. The minimum number of @@ -283,7 +285,7 @@ document.write("Last Published: " + docu mode with multiple servers. Please refer to the ZooKeeper documentation for detail on how to configure ZooKeeper with multiple replicas

    - +

    Running bookies

    To run a bookie, we execute the following command: @@ -341,7 +343,7 @@ document.write("Last Published: " + docu Ideally, /path_to_log_device/ and /path_to_ledger_device/ are each in a different device.

    - +

    ZooKeeper Metadata

    For BookKeeper, we require a ZooKeeper installation to store metadata, and to pass the list Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.pdf (original) and websites/production/zookeeper/content/doc/trunk/bookkeeperConfig.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.html (original) +++ websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + BookKeeper overview @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    BookKeeper overview

    +
    • @@ -230,14 +231,15 @@ document.write("Last Published: " + docu
    +
    - +

    BookKeeper overview

    - +

    BookKeeper introduction

    BookKeeper is a replicated service to reliably log streams of records. In BookKeeper, @@ -287,7 +289,7 @@ document.write("Last Published: " + docu - +

    In slightly more detail...

    BookKeeper implements highly available logs, and it has been designed with write-ahead logging in mind. Besides high availability due to the replicated nature of the service, it provides high throughput due to striping. As we write entries in a subset of bookies of an @@ -308,7 +310,7 @@ document.write("Last Published: " + docu Although two clients may try to recover a ledger concurrently, only one will succeed, the first one that is able to create the close znode for the ledger.

    - +

    Bookkeeper elements and concepts

    BookKeeper uses four basic elements: @@ -361,7 +363,7 @@ document.write("Last Published: " + docu - +

    Bookkeeper initial design

    A set of bookies implements BookKeeper, and we use a quorum-based protocol to replicate data across the bookies. @@ -544,7 +546,7 @@ document.write("Last Published: " + docu invalid (a bad MAC for example), the client will make a request to a different bookie. By using quorum writes, as long as enough bookies are up we are guaranteed to eventually be able to read an entry.

    - +

    Bookkeeper metadata management

    There are some meta data that needs to be made available to BookKeeper clients: @@ -592,7 +594,7 @@ document.write("Last Published: " + docu use znodes to track ledger creation and deletion and also to know the end of the ledger and the bookies that were used to store the ledger. Bookies also watch the ledger list so that they can cleanup ledgers that get deleted.

    - +

    Closing out ledgers

    The process of closing out the ledger and finding the last ledger is difficult due to the durability guarantees of BookKeeper: Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.pdf (original) and websites/production/zookeeper/content/doc/trunk/bookkeeperOverview.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.html (original) +++ websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + BookKeeper Getting Started Guide @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    BookKeeper Getting Started Guide

    +
    • @@ -233,11 +234,12 @@ document.write("Last Published: " + docu
    +
    - +

    Programming with BookKeeper

      @@ -306,7 +308,7 @@ document.write("Last Published: " + docu
    - +

    Instantiating BookKeeper.

    The first step to use BookKeeper is to instantiate a BookKeeper object: @@ -403,7 +405,7 @@ document.write("Last Published: " + docu - +

    Creating a ledger.

    Before writing entries to BookKeeper, it is necessary to create a ledger. With the current BookKeeper API, it is possible to create a ledger both synchronously @@ -549,7 +551,7 @@ document.write("Last Published: " + docu The ctx object passed as a parameter to the call to create a ledger is the one same returned in the callback.

    - +

    Adding entries to a ledger.

    Once we have a ledger handle lh obtained through a call to create a ledger, we @@ -659,7 +661,7 @@ document.write("Last Published: " + docu - +

    Closing a ledger.

    Once a client is done writing, it closes the ledger. The following methods belong @@ -742,7 +744,7 @@ document.write("Last Published: " + docu - +

    Opening a ledger.

    To read from a ledger, a client must open it first. The following methods belong @@ -854,7 +856,7 @@ document.write("Last Published: " + docu - +

    Reading from ledger

    Read calls may request one or more consecutive entries. The following methods belong @@ -968,7 +970,7 @@ document.write("Last Published: " + docu - +

    Deleting a ledger

    Once a client is done with a ledger and is sure that nobody will ever need to read from it again, they can delete the ledger. Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.pdf (original) and websites/production/zookeeper/content/doc/trunk/bookkeeperProgrammer.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.html (original) +++ websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + BookKeeper Getting Started Guide @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    BookKeeper Getting Started Guide

    +
    • @@ -230,11 +231,12 @@ document.write("Last Published: " + docu
    +
    - +

    Getting Started: Setting up BookKeeper to write logs.

    This document contains information to get you started quickly with @@ -243,25 +245,25 @@ document.write("Last Published: " + docu and a simple programming example. For further programming detail, please refer to BookKeeper Programmer's Guide.

    - +

    Pre-requisites

    See System Requirements in the Admin guide.

    - +

    Download

    BookKeeper is distributed along with ZooKeeper. To get a ZooKeeper distribution, download a recent stable release from one of the Apache Download Mirrors.

    - +

    LocalBookKeeper

    Under org.apache.bookkeeper.util, you'll find a java program called LocalBookKeeper.java that sets you up to run BookKeeper on a single machine. This is far from ideal from a performance perspective, but the program is useful for both test and educational purposes.

    - +

    Setting up bookies

    If you're bold and you want more than just running things locally, then you'll need to run bookies in different servers. You'll need at least three bookies @@ -284,7 +286,7 @@ document.write("Last Published: " + docu If we had multiple ZooKeeper servers, this parameter would be a comma separated list of all the hostname:port values corresponding to them.

    - +

    Setting up ZooKeeper

    ZooKeeper stores metadata on behalf of BookKeeper clients and bookies. To get a minimal ZooKeeper installation to work with BookKeeper, we can set up one server running in @@ -326,7 +328,7 @@ document.write("Last Published: " + docu - +

    Example

    In the following excerpt of code, we: Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.pdf (original) and websites/production/zookeeper/content/doc/trunk/bookkeeperStarted.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperStream.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/bookkeeperStream.html (original) +++ websites/production/zookeeper/content/doc/trunk/bookkeeperStream.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + Streaming with BookKeeper @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    Streaming with BookKeeper

    +
    • @@ -216,11 +217,12 @@ document.write("Last Published: " + docu
    +
    - +

    Summary

    @@ -238,7 +240,7 @@ document.write("Last Published: " + docu

    - +

    Writing a stream of bytes

    @@ -419,7 +421,7 @@ document.write("Last Published: " + docu

    - +

    Reading a stream of bytes

    Modified: websites/production/zookeeper/content/doc/trunk/bookkeeperStream.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/bookkeeperStream.pdf (original) and websites/production/zookeeper/content/doc/trunk/bookkeeperStream.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/index.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/index.html (original) +++ websites/production/zookeeper/content/doc/trunk/index.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper: Because Coordinating Distributed Systems is a Zoo @@ -206,6 +206,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper: Because Coordinating Distributed Systems is a Zoo

    +

    ZooKeeper is a high-performance coordination service for distributed applications. It exposes common services - such as Modified: websites/production/zookeeper/content/doc/trunk/index.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/index.pdf (original) and websites/production/zookeeper/content/doc/trunk/index.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/javaExample.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/javaExample.html (original) +++ websites/production/zookeeper/content/doc/trunk/javaExample.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper Java Example @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper Java Example

    +
    • @@ -227,18 +228,19 @@ document.write("Last Published: " + docu
    +
    - +

    A Simple Watch Client

    To introduce you to the ZooKeeper Java API, we develop here a very simple watch client. This ZooKeeper client watches a ZooKeeper node for changes and responds to by starting or stopping a program.

    - +

    Requirements

    The client has four requirements:

      @@ -273,7 +275,7 @@ document.write("Last Published: " + docu

      If the znode disappears, the client kills the executable.

    - +

    Program Design

    Conventionally, ZooKeeper applications are broken into two units, one which maintains the connection, and the other which monitors data. In this application, the class called the Executor @@ -285,7 +287,7 @@ document.write("Last Published: " + docu

    - +

    The Executor Class

    The Executor object is the primary container of the sample application. It contains @@ -431,7 +433,7 @@ public void closing(int rc) {

    - +

    The DataMonitor Class

    @@ -571,7 +573,7 @@ function, when DataMonitor gets an event

    - +

    Complete Source Listings

    Modified: websites/production/zookeeper/content/doc/trunk/javaExample.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/javaExample.pdf (original) and websites/production/zookeeper/content/doc/trunk/javaExample.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/linkmap.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/linkmap.html (original) +++ websites/production/zookeeper/content/doc/trunk/linkmap.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + Site Linkmap Table of Contents @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    Site Linkmap Table of Contents

    +

    This is a map of the complete site and its structure.

    Modified: websites/production/zookeeper/content/doc/trunk/linkmap.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/linkmap.pdf (original) and websites/production/zookeeper/content/doc/trunk/linkmap.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/recipes.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/recipes.html (original) +++ websites/production/zookeeper/content/doc/trunk/recipes.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper Recipes and Solutions @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper Recipes and Solutions

    +
    • @@ -249,12 +250,13 @@ document.write("Last Published: " + docu
    +
    - +

    A Guide to Creating Higher-level Constructs with ZooKeeper

    In this article, you'll find guidelines for using @@ -279,7 +281,7 @@ document.write("Last Published: " + docu as event handles or queues, a more practical means of performing the same function. In general, the examples in this section are designed to stimulate thought.

    - +

    Out of the Box Applications: Name Service, Configuration, Group Membership

    Name service and configuration are two of the primary applications @@ -290,7 +292,7 @@ document.write("Last Published: " + docu group create ephemeral nodes under the group node. Nodes of the members that fail abnormally will be removed automatically when ZooKeeper detects the failure.

    - +

    Barriers

    Distributed systems use barriers to block processing of a set of nodes until a condition is met @@ -334,7 +336,7 @@ document.write("Last Published: " + docu - +

    Double Barriers

    Double barriers enable clients to synchronize the beginning and the end of a computation. When enough processes have joined the barrier, @@ -500,7 +502,7 @@ document.write("Last Published: " + docu (picking the highest for simplicity) to go away. This means that only a single process wakes up on each node deletion except for the last node, which wakes up everyone when it is removed.

    - +

    Queues

    Distributed queues are a common data structure. To implement a distributed queue in ZooKeeper, first designate a znode to hold the queue, @@ -528,7 +530,7 @@ document.write("Last Published: " + docu

    - +

    Priority Queues

    To implement a priority queue, you need only make two simple changes to the generic queue @@ -538,7 +540,7 @@ document.write("Last Published: " + docu from the queue, a client uses an up-to-date children list meaning that the client will invalidate previously obtained children lists if a watch notification triggers for the queue node.

    - +

    Locks

    Fully distributed locks that are globally synchronous, meaning at any snapshot in time no two clients think they hold the same lock. These @@ -637,7 +639,7 @@ document.write("Last Published: " + docu - +

    Shared Locks

    You can implement shared locks by with a few changes to the lock protocol:

    @@ -799,7 +801,7 @@ document.write("Last Published: " + docu - +

    Recoverable Shared Locks

    With minor modifications to the Shared Lock protocol, you make shared locks revocable by modifying the shared lock protocol:

    @@ -821,7 +823,7 @@ document.write("Last Published: " + docu Laser Beams by stipulating in your protocol that the revoker is allowed to delete the lock node if after some length of time the lock isn't deleted by the lock holder.

    - +

    Two-phased Commit

    A two-phase commit protocol is an algorithm that lets all clients in a distributed system agree either to commit a transaction or abort.

    @@ -853,7 +855,7 @@ document.write("Last Published: " + docu

    To address the second problem, you can have the coordinator propagate the transaction to the sites, and have each site creating its own ephemeral node.

    - +

    Leader Election

    A simple way of doing leader election with ZooKeeper is to use the SEQUENCE|EPHEMERAL flags when creating Modified: websites/production/zookeeper/content/doc/trunk/recipes.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/recipes.pdf (original) and websites/production/zookeeper/content/doc/trunk/recipes.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/releasenotes.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/releasenotes.html (original) +++ websites/production/zookeeper/content/doc/trunk/releasenotes.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper 3.0.0 Release Notes @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper 3.0.0 Release Notes

    +
    • @@ -235,6 +236,7 @@ document.write("Last Published: " + docu
    +
    @@ -262,7 +264,7 @@ These release notes include new develope - +

    Migration Instructions when Upgrading to 3.0.0

    @@ -297,7 +299,7 @@ Note: ZooKeeper increments the major ver - +

    Migrating Client Code

    The underlying client-server protocol has changed in version 3.0.0 @@ -306,7 +308,7 @@ Note: ZooKeeper increments the major ver pre-3.0.0 clients are not guaranteed to operate against upgraded 3.0.0 servers and vice-versa).

    - +

    Watch Management

    In previous releases of ZooKeeper any watches registered by clients were lost if the client lost a connection to a ZooKeeper server. @@ -322,7 +324,7 @@ This release also allows clients to spec This gives the developer the ability to modularize logic in different watch functions rather than cramming everything in the watch function attached to the ZooKeeper handle. Call specific watch functions receive all session events for as long as they are active, but will only receive the watch callbacks for which they are registered.

    - +

    Java API

      @@ -344,7 +346,7 @@ Call specific watch functions receive al Also see the current java API

      - +

      C API

        @@ -353,7 +355,7 @@ Also see +

        Migrating Server Data

        The following issues resulted in changes to the on-disk data format (the snapshot and transaction log files contained within the ZK data directory) and require a migration utility to be run. @@ -499,7 +501,7 @@ The following issues resulted in changes

      - +

      Migrating Server Configuration

      There is a significant change to the ZooKeeper server configuration file. @@ -523,7 +525,7 @@ There is a significant change to the Zoo

    - +

    Changes Since ZooKeeper 2.2.1

    Modified: websites/production/zookeeper/content/doc/trunk/releasenotes.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/releasenotes.pdf (original) and websites/production/zookeeper/content/doc/trunk/releasenotes.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/skin/basic.css ============================================================================== --- websites/production/zookeeper/content/doc/trunk/skin/basic.css (original) +++ websites/production/zookeeper/content/doc/trunk/skin/basic.css Mon Aug 8 23:28:06 2011 @@ -39,6 +39,7 @@ img { border: 0; } } .note, .warning, .fixme { + clear:right; border: solid black 1px; margin: 1em 3em; } Modified: websites/production/zookeeper/content/doc/trunk/skin/screen.css ============================================================================== --- websites/production/zookeeper/content/doc/trunk/skin/screen.css (original) +++ websites/production/zookeeper/content/doc/trunk/skin/screen.css Mon Aug 8 23:28:06 2011 @@ -418,12 +418,12 @@ img.cornersmall { position: relative; /* IE bugfix cont'd */ padding: 5px; margin: 0; - width: 45%; + width: 60%; } #footer .lastmodified { position: relative; /* IE bugfix cont'd */ float: right; - width: 45%; + width: 30%; padding: 5px; margin: 0; text-align: right; @@ -455,17 +455,13 @@ acronym { cursor: help; } } #motd-area { - position: relative; /* IE bugfix cont'd */ - float: right; + position:relative; + float:right; width: 35%; background-color: #f0f0ff; - border-top: solid 1px #4C6C8F; - border-bottom: solid 1px #4C6C8F; - margin-bottom: 15px; - margin-left: 15px; - margin-right: 10%; - padding-bottom: 5px; - padding-top: 5px; + border: solid 1px #4C6C8F; + margin: 0px 0px 10px 10px; + padding: 5px; } #minitoc-area { @@ -483,6 +479,10 @@ acronym { cursor: help; } font-weight: normal; } +.abstract{ + text-align:justify; + } + li p { margin: 0; padding: 0; Modified: websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper Administrator's Guide @@ -204,6 +204,7 @@ document.write("Last Published: " + docu

    ZooKeeper Administrator's Guide

    A Guide to Deployment and Administration

    +
    • @@ -324,6 +325,7 @@ document.write("Last Published: " + docu
    +
    @@ -331,7 +333,7 @@ document.write("Last Published: " + docu - +

    Deployment

    This section contains information about deploying Zookeeper and @@ -370,9 +372,9 @@ document.write("Last Published: " + docu section covers situations in which you are setting up ZooKeeper on a limited basis - for evaluation, testing, or development - but not in a production environment.

    - +

    System Requirements

    - +

    Supported Platforms

      @@ -413,7 +415,7 @@ document.write("Last Published: " + docu
    - +

    Required Software

    ZooKeeper runs in Java, release 1.6 or greater (JDK 6 or greater). It runs as an ensemble of @@ -422,7 +424,7 @@ document.write("Last Published: " + docu they run on separate machines. At Yahoo!, ZooKeeper is usually deployed on dedicated RHEL boxes, with dual-core processors, 2GB of RAM, and 80GB IDE hard drives.

    - +

    Clustered (Multi-Server) Setup

    For reliable ZooKeeper service, you should deploy ZooKeeper in a cluster known as an ensemble. As long as a majority @@ -609,7 +611,7 @@ server.3=zoo3:2888:3888 - +

    Single Server and Developer Setup

    If you want to setup ZooKeeper for development purposes, you will probably want to setup a single server instance of ZooKeeper, and then @@ -627,7 +629,7 @@ server.3=zoo3:2888:3888

    - +

    Administration

    This section contains information about running and maintaining @@ -760,7 +762,7 @@ server.3=zoo3:2888:3888 - +

    Designing a ZooKeeper Deployment

    The reliablity of ZooKeeper rests on two basic assumptions.

      @@ -787,7 +789,7 @@ server.3=zoo3:2888:3888 to hold true. Some of these are cross-machines considerations, and others are things you should consider for each and every machine in your deployment.

      - +

      Cross Machine Requirements

      For the ZooKeeper service to be active, there must be a majority of non-failing machines that can communicate with @@ -805,7 +807,7 @@ server.3=zoo3:2888:3888 failure of that switch could cause a correlated failure and bring down the service. The same holds true of shared power circuits, cooling systems, etc.

      - +

      Single Machine Requirements

      If ZooKeeper has to contend with other applications for access to resourses like storage media, CPU, network, or @@ -846,20 +848,20 @@ server.3=zoo3:2888:3888 - +

      Provisioning

      - +

      Things to Consider: ZooKeeper Strengths and Limitations

      - +

      Administering

      - +

      Maintenance

      Little long term maintenance is required for a ZooKeeper cluster however you must be aware of the following:

      - +

      Ongoing Data Directory Cleanup

      The ZooKeeper Data Directory contains files which are a persistent copy @@ -889,7 +891,7 @@ server.3=zoo3:2888:3888 can be run as a cron job on the ZooKeeper server machines to clean up the logs daily.

       java -cp zookeeper.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar:conf org.apache.zookeeper.server.PurgeTxnLog <dataDir> <snapDir> -n <count>
      - +

      Debug Log Cleanup (log4j)

      See the section on logging in this document. It is expected that you will setup a rolling file appender using the @@ -897,7 +899,7 @@ server.3=zoo3:2888:3888 release tar's conf/log4j.properties provides an example of this.

      - +

      Supervision

      You will want to have a supervisory process that manages each of your ZooKeeper server processes (JVM). The ZK server is @@ -916,12 +918,12 @@ server.3=zoo3:2888:3888 examples) managing your ZooKeeper server ensures that if the process does exit abnormally it will automatically be restarted and will quickly rejoin the cluster.

      - +

      Monitoring

      The ZooKeeper service can be monitored in one of two primary ways; 1) the command port through the use of 4 letter words and 2) JMX. See the appropriate section for your environment/requirements.

      - +

      Logging

      ZooKeeper uses log4j version 1.2 as its logging infrastructure. The ZooKeeper default log4j.properties @@ -931,7 +933,7 @@ server.3=zoo3:2888:3888

      For more information, see Log4j Default Initialization Procedure of the log4j manual.

      - +

      Troubleshooting

      @@ -951,7 +953,7 @@ server.3=zoo3:2888:3888
      - +

      Configuration Parameters

      ZooKeeper's behavior is governed by the ZooKeeper configuration file. This file is designed so that the exact same file can be used by @@ -959,7 +961,7 @@ server.3=zoo3:2888:3888 layouts are the same. If servers use different configuration files, care must be taken to ensure that the list of servers in all of the different configuration files match.

      - +

      Minimum Configuration

      Here are the minimum configuration keywords that must be defined in the configuration file:

      @@ -1006,7 +1008,7 @@ server.3=zoo3:2888:3888 - +

      Advanced Configuration

      The configuration settings in the section are optional. You can use them to further fine tune the behaviour of your ZooKeeper servers. @@ -1102,7 +1104,7 @@ server.3=zoo3:2888:3888 level) that a single client, identified by IP address, may make to a single member of the ZooKeeper ensemble. This is used to prevent certain classes of DoS attacks, including file - descriptor exhaustion. The default is 10. Setting this to 0 + descriptor exhaustion. The default is 60. Setting this to 0 entirely removes the limit on concurrent connections.

      @@ -1148,7 +1150,7 @@ server.3=zoo3:2888:3888 - +

      Cluster Options

      The options in this section are designed for use with an ensemble of servers -- that is, when deploying clusters of servers.

      @@ -1305,7 +1307,7 @@ server.3=zoo3:2888:3888

      - +

      Authentication & Authorization Options

      The options in this section allow control over authentication/authorization performed by the service.

      @@ -1339,7 +1341,7 @@ server.3=zoo3:2888:3888 - +

      Unsafe Options

      The following options can be useful, but be careful when you use them. The risk of each is explained along with the explanation of what @@ -1384,7 +1386,7 @@ server.3=zoo3:2888:3888 - +

      Communication using the Netty framework

      New in @@ -1417,7 +1419,7 @@ server.3=zoo3:2888:3888

      TBD - how to manage certificates

      - +

      ZooKeeper Commands: The Four Letter Words

      ZooKeeper responds to a small set of commands. Each command is composed of four letters. You issue the commands to ZooKeeper via telnet @@ -1598,7 +1600,7 @@ zk_max_file_descriptor_count 1024 - on 

      $ echo ruok | nc 127.0.0.1 5111
 imok
      - +

      Data File Management

      ZooKeeper stores its data in a data directory and its transaction log in a transaction log directory. By default these two directories are @@ -1606,7 +1608,7 @@ imok transaction log files in a separate directory than the data files. Throughput increases and latency decreases when transaction logs reside on a dedicated log devices.

      - +

      The Data Directory

      This directory has two files in it:

        @@ -1652,14 +1654,14 @@ imok idempotent nature of its updates. By replaying the transaction log against fuzzy snapshots ZooKeeper gets the state of the system at the end of the log.

        - +

        The Log Directory

        The Log Directory contains the ZooKeeper transaction logs. Before any update takes place, ZooKeeper ensures that the transaction that represents the update is written to non-volatile storage. A new log file is started each time a snapshot is begun. The log file's suffix is the first zxid written to that log.

        - +

        File Management

        The format of snapshot and log files does not change between standalone ZooKeeper servers and different configurations of @@ -1679,7 +1681,7 @@ imok this document for more details on setting a retention policy and maintenance of ZooKeeper storage.

        - +

        Things to Avoid

        Here are some common problems you can avoid by configuring ZooKeeper correctly:

        @@ -1733,7 +1735,7 @@ imok - +

        Best Practices

        For best results, take note of the following list of good Zookeeper practices:

        Modified: websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperAdmin.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + Introduction to hierarchical quorums @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    Introduction to hierarchical quorums

    +
    Modified: websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperHierarchicalQuorums.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/zookeeperInternals.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperInternals.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperInternals.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper Internals @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper Internals

    +
    • @@ -249,12 +250,13 @@ document.write("Last Published: " + docu
    +
    - +

    Introduction

    This document contains information on the inner workings of ZooKeeper. @@ -278,12 +280,12 @@ document.write("Last Published: " + docu

    - +

    Atomic Broadcast

    At the heart of ZooKeeper is an atomic messaging system that keeps all of the servers in sync.

    - +

    Guarantees, Properties, and Definitions

    The specific guarantees provided by the messaging system used by ZooKeeper are the following:

    @@ -472,7 +474,7 @@ yourself, Does the set of proposals included all the proposals that really have been committed? The answer is yes. Below, we make clear why.)

    - +

    Leader Activation

    Leader activation includes leader election. We currently have two leader election @@ -558,7 +560,7 @@ NEW_LEADER proposal will not be committe When this happens, the leader and any remaining followers will timeout and go back to leader election.

    - +

    Active Messaging

    Leader Activation does all the heavy lifting. Once the leader is coronated he can @@ -606,7 +608,7 @@ message when that proposal is committed. - +

    Summary

    So there you go. Why does it work? Specifically, why does is set of proposals believed by a new leader always contain any proposal that has actually been committed? @@ -618,7 +620,7 @@ follow a single leader at a time; a new from the previous epoch since it has seen the highest zxid from a quorum of servers; any uncommited proposals from a previous epoch seen by a new leader will be committed by that leader before it becomes active.

    - +

    Comparisons

    Isn't this just Multi-Paxos? No, Multi-Paxos requires some way of assuring that @@ -638,7 +640,7 @@ proposals and to not worry about duplica

    - +

    Quorums

    @@ -673,7 +675,7 @@ hierarchy of groups.

    - +

    Logging

    @@ -689,14 +691,14 @@ of the Zoo

    - +

    Developer Guidelines

    Please follow the slf4j manual when creating log statements within code. Also read the FAQ on performance , when creating log statements. Patch reviewers will look for the following:

    - +

    Logging at the Right Level

    There are several levels of logging in slf4j. @@ -727,7 +729,7 @@ It's important to pick the right one. In

    ZooKeeper is typically run in production such that log messages of INFO level severity and higher (more severe) are output to the log.

    - +

    Use of Standard slf4j Idioms

    Static Message Logging Modified: websites/production/zookeeper/content/doc/trunk/zookeeperInternals.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperInternals.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperInternals.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/zookeeperJMX.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperJMX.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperJMX.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper JMX @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper JMX

    +
    • @@ -219,12 +220,13 @@ document.write("Last Published: " + docu
    +
    - +

    JMX

    Apache ZooKeeper has extensive support for JMX, allowing you @@ -243,7 +245,7 @@ document.write("Last Published: " + docu

    - +

    Starting ZooKeeper with JMX enabled

    The class @@ -256,7 +258,7 @@ document.write("Last Published: " + docu

    - +

    Run a JMX console

    There are a number of JMX consoles available which can connect @@ -291,7 +293,7 @@ document.write("Last Published: " + docu

    - +

    ZooKeeper MBean Reference

    This table details JMX for a server participating in a Modified: websites/production/zookeeper/content/doc/trunk/zookeeperJMX.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperJMX.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperJMX.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/zookeeperObservers.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperObservers.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperObservers.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper Observers @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper Observers

    +
    • @@ -217,12 +218,13 @@ document.write("Last Published: " + docu
    +
    - +

    Observers: Scaling ZooKeeper Without Hurting Write Performance

    @@ -260,7 +262,7 @@ document.write("Last Published: " + docu

    - +

    How to use Observers

    Setting up a ZooKeeper ensemble that uses Observers is very simple, @@ -295,7 +297,7 @@ document.write("Last Published: " + docu

    - +

    Example use cases

    Modified: websites/production/zookeeper/content/doc/trunk/zookeeperObservers.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperObservers.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperObservers.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/zookeeperOver.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperOver.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperOver.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper @@ -203,6 +203,7 @@ document.write("Last Published: " + docu PDF

    ZooKeeper

    +
    • @@ -246,12 +247,13 @@ document.write("Last Published: " + docu
    +
    - +

    ZooKeeper: A Distributed Coordination Service for Distributed Applications

    @@ -266,7 +268,7 @@ document.write("Last Published: " + docu especially prone to errors such as race conditions and deadlock. The motivation behind ZooKeeper is to relieve distributed applications the responsibility of implementing coordination services from scratch.

    - +

    Design Goals

    ZooKeeper is simple. ZooKeeper @@ -319,7 +321,7 @@ document.write("Last Published: " + docu especially fast in "read-dominant" workloads. ZooKeeper applications run on thousands of machines, and it performs best where reads are more common than writes, at ratios of around 10:1.

    - +

    Data model and the hierarchical namespace

    The name space provided by ZooKeeper is much like that of a standard file system. A name is a sequence of path elements separated by @@ -337,7 +339,7 @@ document.write("Last Published: " + docu - +

    Nodes and ephemeral nodes

    Unlike is standard file systems, each node in a ZooKeeper namespace can have data associated with it as well as children. It is @@ -360,7 +362,7 @@ document.write("Last Published: " + docu exists as long as the session that created the znode is active. When the session ends the znode is deleted. Ephemeral nodes are useful when you want to implement [tbd].

    - +

    Conditional updates and watches

    ZooKeeper supports the concept of watches. Clients can set a watch on a znodes. A watch will be triggered and @@ -369,7 +371,7 @@ document.write("Last Published: " + docu connection between the client and one of the Zoo Keeper servers is broken, the client will receive a local notification. These can be used to [tbd].

    - +

    Guarantees

    ZooKeeper is very fast and very simple. Since its goal, though, is to be a basis for the construction of more complicated services, such as @@ -423,7 +425,7 @@ document.write("Last Published: " + docu

    For more information on these, and how they can be used, see [tbd]

    - +

    Simple API

    One of the design goals of ZooKeeper is provide a very simple programming interface. As a result, it supports only these @@ -490,7 +492,7 @@ document.write("Last Published: " + docu to implement higher level operations, please refer to [tbd]

    - +

    Implementation

    ZooKeeper Components shows the high-level components @@ -532,7 +534,7 @@ document.write("Last Published: " + docu calculates what the state of the system is when the write is to be applied and transforms this into a transaction that captures this new state.

    - +

    Uses

    The programming interface to ZooKeeper is deliberately simple. With it, however, you can implement higher order operations, such as @@ -541,7 +543,7 @@ document.write("Last Published: " + docu white paper and video presentation.] For more information, see [tbd]

    - +

    Performance

    ZooKeeper is designed to be highly performant. But is it? The results of the ZooKeeper's development team at Yahoo! Research indicate @@ -619,7 +621,7 @@ document.write("Last Published: " + docu - +

    Reliability

    To show the behavior of the system over time as failures are injected we ran a ZooKeeper service made up of @@ -648,7 +650,7 @@ document.write("Last Published: " + docu ZooKeeper takes less than 200ms to elect a new leader. Third, as followers recover, ZooKeeper is able to raise throughput again once they start processing requests.

    - +

    The ZooKeeper Project

    ZooKeeper has been Modified: websites/production/zookeeper/content/doc/trunk/zookeeperOver.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperOver.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperOver.pdf Mon Aug 8 23:28:06 2011 differ Modified: websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.html ============================================================================== --- websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.html (original) +++ websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.html Mon Aug 8 23:28:06 2011 @@ -3,7 +3,7 @@ - + ZooKeeper Programmer's Guide @@ -204,6 +204,7 @@ document.write("Last Published: " + docu

    ZooKeeper Programmer's Guide

    Developing Distributed Applications that use ZooKeeper

    +
    • @@ -323,6 +324,7 @@ document.write("Last Published: " + docu
    +
    @@ -330,7 +332,7 @@ document.write("Last Published: " + docu -     +

    Introduction

    This document is a guide for developers wishing to create @@ -433,7 +435,7 @@ document.write("Last Published: " + docu

    - +

    The ZooKeeper Data Model

    ZooKeeper has a hierarchal name space, much like a distributed file @@ -488,7 +490,7 @@ document.write("Last Published: " + docu - +

    ZNodes

    Every node in a ZooKeeper tree is referred to as a znode. Znodes maintain a stat structure that @@ -520,14 +522,14 @@ document.write("Last Published: " + docu

    Znodes are the main enitity that a programmer access. They have several characteristics that are worth mentioning here.

    - +

    Watches

    Clients can set watches on znodes. Changes to that znode trigger the watch and then clear the watch. When a watch triggers, ZooKeeper sends the client a notification. More information about watches can be found in the section ZooKeeper Watches.

    - +

    Data Access

    The data stored at each znode in a namespace is read and written atomically. Reads get all the data bytes associated with a znode and a @@ -547,13 +549,13 @@ document.write("Last Published: " + docu large data storage is needed, the usually pattern of dealing with such data is to store it on a bulk storage system, such as NFS or HDFS, and store pointers to the storage locations in ZooKeeper.

    - +

    Ephemeral Nodes

    ZooKeeper also has the notion of ephemeral nodes. These znodes exists as long as the session that created the znode is active. When the session ends the znode is deleted. Because of this behavior ephemeral znodes are not allowed to have children.

    - +

    Sequence Nodes -- Unique Naming

    When creating a znode you can also request that ZooKeeper append a monotonically increasing counter to the end @@ -567,7 +569,7 @@ document.write("Last Published: " + docu (4bytes) maintained by the parent node, the counter will overflow when incremented beyond 2147483647 (resulting in a name "<path>-2147483647").

    - +

    Time in ZooKeeper

    ZooKeeper tracks time multiple ways:

      @@ -636,7 +638,7 @@ document.write("Last Published: " + docu
    - +

    ZooKeeper Stat Structure

    The Stat structure for each znode in ZooKeeper is made up of the following fields:

    @@ -771,7 +773,7 @@ document.write("Last Published: " + docu - +

    ZooKeeper Sessions

    A ZooKeeper client establishes a session with the ZooKeeper @@ -953,7 +955,7 @@ document.write("Last Published: " + docu

    - +

    ZooKeeper Watches

    All of the read operations in ZooKeeper - getData(), getChildren(), and exists() - have the option of setting a watch as a @@ -1036,7 +1038,7 @@ document.write("Last Published: " + docu general this all occurs transparently. There is one case where a watch may be missed: a watch for the existance of a znode not yet created will be missed if the znode is created and deleted while disconnected.

    - +

    What ZooKeeper Guarantees about Watches

    With regard to watches, ZooKeeper maintains these guarantees:

    @@ -1071,7 +1073,7 @@ document.write("Last Published: " + docu - +

    Things to Remember about Watches

      @@ -1130,7 +1132,7 @@ document.write("Last Published: " + docu
    - +

    ZooKeeper access control using ACLs

    ZooKeeper uses ACLs to control access to its znodes (the @@ -1165,7 +1167,7 @@ document.write("Last Published: " + docu example, the pair (ip:19.22.0.0/16, READ) gives the READ permission to any clients with an IP address that starts with 19.22.

    - +

    ACL Permissions

    ZooKeeper supports the following permissions:

      @@ -1221,7 +1223,7 @@ document.write("Last Published: " + docu node, but nothing more. (The problem is, if you want to call zoo_exists() on a node that doesn't exist, there is no permission to check.)

      - +

      Builtin ACL Schemes

      ZooKeeeper has the following built in schemes:

        @@ -1270,7 +1272,7 @@ document.write("Last Published: " + docu
      - +

      ZooKeeper C client API

      The following constants are provided by the ZooKeeper C library:

      @@ -1492,7 +1494,7 @@ int main(int argc, char argv) {
    - +

    Pluggable ZooKeeper authentication

    ZooKeeper runs in a variety of different environments with @@ -1578,7 +1580,7 @@ authProvider.2=com.f.MyAuth2

    - +

    Consistency Guarantees

    ZooKeeper is a high performance, scalable service. Both reads and @@ -1704,12 +1706,12 @@ authProvider.2=com.f.MyAuth2

    - +

    Bindings

    The ZooKeeper client libraries come in two languages: Java and C. The following sections describe these.

    - +

    Java Binding

    There are two packages that make up the ZooKeeper Java binding: org.apache.zookeeper and org.apache.zookeeper.data. The rest of the @@ -1776,7 +1778,7 @@ authProvider.2=com.f.MyAuth2 (SESSION_EXPIRED and AUTH_FAILED), the ZooKeeper object becomes invalid. On a close, the two threads shut down and any further access on zookeeper handle is undefined behavior and should be avoided.

    - +

    C Binding

    The C binding has a single-threaded and multi-threaded library. The multi-threaded library is easiest to use and is most similar to the @@ -1793,7 +1795,7 @@ authProvider.2=com.f.MyAuth2 (i.e. FreeBSD 4.x). In all other cases, application developers should link with zookeeper_mt, as it includes support for both Sync and Async API.

    - +

    Installation

    If you're building the client from a check-out from the Apache repository, follow the steps outlined below. If you're building from a @@ -1924,7 +1926,7 @@ authProvider.2=com.f.MyAuth2 - +

    Using the C Client

    You can test your client by running a ZooKeeper server (see instructions on the project wiki page on how to run it) and connecting @@ -1982,7 +1984,7 @@ authProvider.2=com.f.MyAuth2

    - +

    Building Blocks: A Guide to ZooKeeper Operations

    This section surveys all the operations a developer can perform @@ -2000,28 +2002,28 @@ authProvider.2=com.f.MyAuth2 - +

    Handling Errors

    Both the Java and C client bindings may report errors. The Java client binding does so by throwing KeeperException, calling code() on the exception will return the specific error code. The C client binding returns an error code as defined in the enum ZOO_ERRORS. API callbacks indicate result code for both language bindings. See the API documentation (javadoc for Java, doxygen for C) for full details on the possible errors and their meaning.

    - +

    Connecting to ZooKeeper

    - +

    Read Operations

    - +

    Write Operations

    - +

    Handling Watches

    - +

    Miscelleaneous ZooKeeper Operations

    - +

    Program Structure, with Simple Example

    @@ -2030,7 +2032,7 @@ authProvider.2=com.f.MyAuth2

    - +

    Gotchas: Common Problems and Troubleshooting

    So now you know ZooKeeper. It's fast, simple, your application Modified: websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.pdf ============================================================================== Files websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.pdf (original) and websites/production/zookeeper/content/doc/trunk/zookeeperProgrammers.pdf Mon Aug 8 23:28:06 2011 differ