hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/SoC2010Ideas" by PatrickHunt
Date Thu, 11 Mar 2010 20:26:48 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/SoC2010Ideas" page has been changed by PatrickHunt.


  Java, some networking familiarity
  ==== Description ====
  ZK 3.3.0 added ''observers'' which are non-voting members of a ZK ensemble. One use case
for observers is as a proxy to a remote voting ensemble, say in a different data center. Since
observers do not need to vote, there are less strict latency requirements on the delivery
of messages to them. WAN traffic is also expensive. This project would investigate and implement
batching of messages to observers, and potential mechanisms for decreasing the number of messages
that need to be sent. For example, a destructive update to a znode twice in a row does not
theoretically need to be sent twice - although making this work correctly with ZAB will be
a challenge.  
+ === FUSE module for BookKeeper ===
+ ==== Possible Mentor ====
+ Ben Reed (breed at apache dot org) & Patrick Hunt (phunt at apache dot org)
+ ==== Requirements ====
+ C/Java, some networking familiarity
+ ==== Description ====
+ BookKeeper is a distributed write ahead log with client & server written in Java. BookKeeper
client & server also use ZooKeeper. There is a BookKeeper API that clients can use to
integrate write ahead logging into their application. It would be a lot easier if applications
could use BK without changes to the client application through use of a file system api (FUSE).
The project would involve implementing a C interface for BookKeeper (Java already exists)
and implementing the FUSE module.
+ Example use: the write ahead log in mysql, called binlogs are typically written to the local
filesystem using the std filesystem api. We could modify mysql to use BooKeeper, however if
we had a BK FUSE module we could run it (mysql) w/o any modification and get the performance/reliability
of a distributed write ahead log.

View raw message