hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/GSoCReadOnlyMode" by Sergey Doroshenko
Date Fri, 14 May 2010 18:58:30 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/GSoCReadOnlyMode" page has been changed by Sergey Doroshenko.


  '''Detailed description'''
- Currently when ZK looses a quorum, say 3 of 5 servers become dead, 2 others which are alive
+ <<BR>>
+  * New client class for read-only mode will be created by extending !ZooKeeper class.<<BR>>When
this client is connected to a server-in-majority, it should behave as a normal one. But when
server is partitioned, client will allow only read requests, write requests will fail on the
-  1. start throwing exceptions during leader election
-  1. drop existing clients connections
-  1. drop new clients should ones connect
- -- and so and so in a loop, until quorum will be re-established.
+  * How application will know mode's changed?<<BR>>Global callback for read-only
client will inform application about mode change from usual to read-only and vice versa, besides
current notifications like connection loss.<<BR>>
- Naive idea is to change current behaviour in the way that server doesn't drop clients when
partitioned (only informs them about partitioning), and only throw exceptions when client
is attempting a write. This is backwards incompatible change on how users currently manage
+  * Make server distinguish these two types of clients: add "am-i-readonly-client" field
to a packet client sends to a server.  <<BR>>This will involve changes in both
Java and C clients.<<BR>>
+  * Currently, server still accepts connections even if it's partitioned, but drops them
quickly after they get connected.<<BR>>This should be changed in this way: when
server loses a quorum, it doesn't drop read-only clients. It just informs them about mode
change (they will receive notification via their global watcher). After this it should respond
to read requests and throw exceptions to write requests.<<BR>>(For normal not
read-only clients, server will behave as it does now.)<<BR>>This will cause changes
in Leader and Follower/Ovserver classes (and corresponding subclasses of !ZooKeeperServer),
since currently they just nullify !ZooKeeperServer when they lose a qourum.<<BR>>
+  * Recovering from partitioning has two aspects:<<BR>>
+   * On the server side, when server regains a quorum, it should notify all currently connected
read-only clients about mode change; after this it can answer to write requests.
+   * On the client side, read-only client should keep hunting for a server-in-majority, and
if one found, it should reconnect to it, and fire global watcher that mode was changed.
- My idea is to do the following:
+ Application developers will decide which client to use, i.e. they'll choose whether to have
guaranteed consistent view of system, or agree to sometimes have outdated view in return for
read access.
-  * Create "read-only" Zookeeper client -- either by adding read-only mode in current !ZooKeeper
class (and passing "mode" parameter in ctor), or by creating separate class (possibly by extending
current !ZooKeeper to allow transparent usage by ZK users).
-  * Make server distinguish these 2 types of clients. For old ones behaviour remains the
same, which eliminates backwards incompatibility. <<BR>>For new "read-only" clients
server actions are these: <<BR>>
-   * when we are in quorum, allow all operations, as if the client is the normal one
-   * when partitioning occurs, inform clients about state change
-   * when we are partitioned, allow only read operations, answer with exception to writes
+ Enabling read-only mode in current applications will involve changing session handling logic
(they will have to detect new "mode changed" notifications), but since read-only client is
a subclass of current client, interface remains unchanged and transition should be very transparent.
+ It's important to note, despite server side will be (heavily?) changed, behavior for usual
clients remains the same, so there will be no backwards incompatibility issues introduced.
+ <<BR>>
  Benefits of this approach: transparent usage of a new client, backwards compatible.

View raw message