hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Trivial Update of "ZooKeeper/Tutorial" by jeremyhanna
Date Thu, 09 Dec 2010 23:39:32 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/Tutorial" page has been changed by jeremyhanna.
The comment on this change is: Removed ZooKeeper keyword links and fixed a couple of typos..
http://wiki.apache.org/hadoop/ZooKeeper/Tutorial?action=diff&rev1=3&rev2=4

--------------------------------------------------

  = Programming with ZooKeeper - A quick tutorial =
  
- In this tutorial, we show simple implementations of barriers and producer-consumer queues
using ZooKeeper. We call the respective classes Barrier and Queue. These examples assume that
you have at least one ZooKeeper server running.
+ In this tutorial, we show simple implementations of barriers and producer-consumer queues
using !ZooKeeper. We call the respective classes Barrier and Queue. These examples assume
that you have at least one !ZooKeeper server running.
  
  Both primitives use the following common excerpt of code:
  
@@ -30, +30 @@

  
  }}}
  
- Both classes extend SyncPrimitive. In this way, we execute steps that are common to all
primitives in the constructor of SyncPrimitive. To keep the examples simple, we create a ZooKeeper
object the first time we instantiate either a barrier object or a queue object, and we declare
a static variable that is a reference to this object. The subsequent instances of Barrier
and Queue check whether a ZooKeeper object exists. Alternatively, we could have the application
creating a ZooKeeper object and passing it to the constructor of Barrier and Queue.
+ Both classes extend !SyncPrimitive. In this way, we execute steps that are common to all
primitives in the constructor of !SyncPrimitive. To keep the examples simple, we create a
!ZooKeeper object the first time we instantiate either a barrier object or a queue object,
and we declare a static variable that is a reference to this object. The subsequent instances
of Barrier and Queue check whether a !ZooKeeper object exists. Alternatively, we could have
the application creating a !ZooKeeper object and passing it to the constructor of Barrier
and Queue.
  
- We use the process() method to process notifications triggered due to watches. In the following
discussion, we present code that sets watches. A watch is internal structure that enables
ZooKeeper to notify a client of a change to a node. For example, if a client is waiting for
other clients to leave a barrier, then it can set a watch and wait for modifications to a
particular node, which can indicate that it is the end of the wait. This point becomes clear
once we go over the examples.
+ We use the process() method to process notifications triggered due to watches. In the following
discussion, we present code that sets watches. A watch is internal structure that enables
!ZooKeeper to notify a client of a change to a node. For example, if a client is waiting for
other clients to leave a barrier, then it can set a watch and wait for modifications to a
particular node, which can indicate that it is the end of the wait. This point becomes clear
once we go over the examples.
  
  == Barriers ==
  A barrier is a primitive that enables a group of processes to synchronize the beginning
and the end of a computation. The general idea of this implementation is to have a barrier
node that serves the purpose of being a parent for individual process nodes. Suppose that
we call the barrier node "/b1". Each process "p" then creates a node "/b1/p". Once enough
processes have created their corresponding nodes, joined processes can start the computation.
  
  In this example, each process instantiates a Barrier object, and its constructor takes as
parameters:
  
-     * The address of a ZooKeeper server (e.g., "zoo1.foo.com:2181");
+     * The address of a !ZooKeeper server (e.g., "zoo1.foo.com:2181");
-     * The path of the barrier node on ZooKeeper (e.g., "/b1");
+     * The path of the barrier node on !ZooKeeper (e.g., "/b1");
      * The size of the group of processes.
  
  
- The constructor of Barrier passes the address of the Zookeeper server to the constructor
of the parent class. The parent class creates a ZooKeeper instance if one does not exist.
The constructor of Barrier then creates a barrier node on ZooKeeper, which is the parent node
of all process nodes, and we call root (obs: this is not the ZooKeeper root "/").
+ The constructor of Barrier passes the address of the Zookeeper server to the constructor
of the parent class. The parent class creates a !ZooKeeper instance if one does not exist.
The constructor of Barrier then creates a barrier node on !ZooKeeper, which is the parent
node of all process nodes, and we call root (obs: this is not the !ZooKeeper root "/").
  
  {{{
        /**
@@ -102, +102 @@

          }
  }}}
  
- Note that enter() throws both KeeperException and InterruptedException, so it is reponsability
of the application to catch and handle such exceptions.
+ Note that enter() throws both !KeeperException and !InterruptedException, so it is reponsibility
of the application to catch and handle such exceptions.
  
  
- Once the computation is finished, a process calls leave() to leave the barrier. First it
deletes its corresponding node, and then it gets the children of the root node. If there is
at least one child, then it waits for a notification (obs: note that the second parameter
of the call to getChildren() is true, meaning that ZooKeeper has to set a watch on the the
root node). Upon reception of a notification, it checks once more whether the root node has
any child.
+ Once the computation is finished, a process calls leave() to leave the barrier. First it
deletes its corresponding node, and then it gets the children of the root node. If there is
at least one child, then it waits for a notification (obs: note that the second parameter
of the call to getChildren() is true, meaning that !ZooKeeper has to set a watch on the the
root node). Upon reception of a notification, it checks once more whether the root node has
any child.
  
  {{{
       /**
@@ -133, +133 @@

  
  == Producer-Consumer Queues ==
  
- A producer-consumer queue is a distributed data estructure thata group of processes use
to generate and consume items. Producer processes create new elements and add them to the
queue. Consumer processes remove elements from the list, and process them. In this implementation,
the elements are simple integers. The queue is represented by a root node, and to add an element
to the queue, a producer process creates a new node, a child of the root node.
+ A producer-consumer queue is a distributed data structure that group of processes use to
generate and consume items. Producer processes create new elements and add them to the queue.
Consumer processes remove elements from the list, and process them. In this implementation,
the elements are simple integers. The queue is represented by a root node, and to add an element
to the queue, a producer process creates a new node, a child of the root node.
  
- The following excerpt of code corresponds to the constructor of the object. As with Barrier
objects, it first calls the constructor of the parent class, SyncPrimitive, that creates a
ZooKeeper object if one doesn't exist. It then verifies if the root node of the queue exists,
and creates if it doesn't.
+ The following excerpt of code corresponds to the constructor of the object. As with Barrier
objects, it first calls the constructor of the parent class, !SyncPrimitive, that creates
a !ZooKeeper object if one doesn't exist. It then verifies if the root node of the queue exists,
and creates if it doesn't.
  
  {{{
       /**
@@ -158, +158 @@

          }
  }}}
  
- A producer process calls "produce()" to add an element to the queue, and passes an integer
as an argument. To add an element to the queue, the method creates a new node using "create()",
and uses the SEQUENCE flag to instruct ZooKeeper to append the value of the sequencer counter
associated to the root node. In this way, we impose a total order on the elements of the queue,
thus guaranteeing that the oldest element of the queue is the next one consumed.
+ A producer process calls "produce()" to add an element to the queue, and passes an integer
as an argument. To add an element to the queue, the method creates a new node using "create()",
and uses the SEQUENCE flag to instruct !ZooKeeper to append the value of the sequencer counter
associated to the root node. In this way, we impose a total order on the elements of the queue,
thus guaranteeing that the oldest element of the queue is the next one consumed.
  
  
  {{{

Mime
View raw message