hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Robinson (JIRA)" <j...@apache.org>
Subject [jira] Updated: (ZOOKEEPER-368) Observers
Date Mon, 13 Jul 2009 14:49:14 GMT

     [ https://issues.apache.org/jira/browse/ZOOKEEPER-368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Henry Robinson updated ZOOKEEPER-368:

    Attachment: ZOOKEEPER-368.patch

Here is a new patch. The headline changes are:

* Observers now commit proposals to disk
* Observers can issue proposals
* Clients can connect to observers as though they were followers.

I have moved around a lot of code, as follows:

* Observers are now in their own class, and along with Followers are a subtype of a new class
* Peer contains common code shared between observers and followers.
* In particular, followLeader has been broken up into several methods so that it can be shared
between observers and followers. The code has barely changed, only the partitioning. I think
this helps its readability as well.
* There is a new ObserverZooKeeperServer class, which along with FollowerZooKeeper class is
a subtype of a new class PeerZooKeeperServer. Again, this is for code reuse purposes.

All Java tests pass, and the patch applies cleanly for me against trunk. I haven't included
any new tests, which are the next thing I will do. As far as I am aware, this patch is feature

> Observers
> ---------
>                 Key: ZOOKEEPER-368
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-368
>             Project: Zookeeper
>          Issue Type: New Feature
>          Components: quorum
>            Reporter: Flavio Paiva Junqueira
>            Assignee: Henry Robinson
>         Attachments: ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch
> Currently, all servers of an ensemble participate actively in reaching agreement on the
order of ZooKeeper transactions. That is, all followers receive proposals, acknowledge them,
and receive commit messages from the leader. A leader issues commit messages once it receives
acknowledgments from a quorum of followers. For cross-colo operation, it would be useful to
have a third role: observer. Using Paxos terminology, observers are similar to learners. An
observer does not participate actively in the agreement step of the atomic broadcast protocol.
Instead, it only commits proposals that have been accepted by some quorum of followers.
> One simple solution to implement observers is to have the leader forwarding commit messages
not only to followers but also to observers, and have observers applying transactions according
to the order followers agreed upon. In the current implementation of the protocol, however,
commit messages do not carry their corresponding transaction payload because all servers different
from the leader are followers and followers receive such a payload first through a proposal
message. Just forwarding commit messages as they currently are to an observer consequently
is not sufficient. We have a couple of options:
> 1- Include the transaction payload along in commit messages to observers;
> 2- Send proposals to observers as well.
> Number 2 is simpler to implement because it doesn't require changing the protocol implementation,
but it increases traffic slightly. The performance impact due to such an increase might be
insignificant, though.
> For scalability purposes, we may consider having followers also forwarding commit messages
to observers. With this option, observers can connect to followers, and receive messages from
followers. This choice is important to avoid increasing the load on the leader with the number
of observers. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message