hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/ConnectionStrategy" by PatrickHunt
Date Wed, 26 May 2010 17:49:54 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/ConnectionStrategy" page has been changed by PatrickHunt.


New page:
= Connection strategy design for ZooKeeper client API =

A connection strategy allows control over the way that ZooKeeper clients (we would implement
this for both c and java apis) connect to a serving ensemble. Today we have two strategies,
randomized round robin (default) and ordered round robin, both of which are hard coded into
the client implementation.

This page details a design which adds a connection "strategy" to the ZooKeeper client API.
By strategy I mean something like this: http://en.wikipedia.org/wiki/Strategy_pattern which
would allow developers using the ZooKeeper client API to reuse existing strategies or specify
their own.

See [[https://issues.apache.org/jira/browse/ZOOKEEPER-779][ZOOKEEPER-779]] for some background
on this.

== Use cases ==

Today's "round robin" connection strategy works well in most cases, however there are some
use cases where it fails:

 1. balancing the load on a particular server of the ensemble. Round robin does a decent job
on this, however there are cases where it fails - for example when a single server is restarted.
 1. choosing the "closest" server. Typically not an issue, except for the case where the ensemble
itself is distributed over high latency (WAN) connections.
 1. exposing ensemble availability to the client code. In some cases the client would like
to fall back to a hard coded state if it cannot reach the ensemble.

== Non Goals ==

 1. This document does not attempt to address how the client gets the list of servers making
up the ensemble, only how it determines which server to connect to.

== Goals == 

 1. allow the client to select a strategy at runtime
 1. allow reuse of strategy code
 1. allow users to create their own strategy, or extend existing strategies
 1. strategy API should allow
  1. setStrategy(strat) on ZooKeeper client object, default should be RRR
  1. more here ... which will require real thought/consideration.
  1. provide some sort of context to the strat (list of servers for example, what more?)
  1. expectation is the strat will make the connection to the server and hand back to ZooKeeper
client code
  1. strat may maintain multiple connections, might want to force ZooKeeper to disconnect
and reconnect to another server
   1. this is the case for example where you connected to a "far away" server because close-in
was down, but then later close in server becomes available again and you prefer to use that.
similar for balancing.

View raw message