hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Reed (JIRA)" <j...@apache.org>
Subject [jira] Commented: (ZOOKEEPER-368) Observers
Date Wed, 11 Nov 2009 17:27:39 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776524#action_12776524
] 

Benjamin Reed commented on ZOOKEEPER-368:
-----------------------------------------

nice reviewer guide! the patch looks really good. for me it's good to go once you have address
the 4 things that flavio raised. (the forest doc is a pain, if you have troubles with it i'll
help with the formatting if you give me the content.)

for historical purposes do you have a copy of that summary that was produced of the differences
in operation and motivation between zero-weight followers and observers? it's been a while
and i can't remember how it was published. it would be good to put a comment about it here
in this issue.

> Observers
> ---------
>
>                 Key: ZOOKEEPER-368
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-368
>             Project: Zookeeper
>          Issue Type: New Feature
>          Components: quorum
>            Reporter: Flavio Paiva Junqueira
>            Assignee: Henry Robinson
>         Attachments: obs-refactor.patch, observer-refactor.patch, observers sync benchmark.png,
observers.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch,
ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch, ZOOKEEPER-368.patch
>
>
> Currently, all servers of an ensemble participate actively in reaching agreement on the
order of ZooKeeper transactions. That is, all followers receive proposals, acknowledge them,
and receive commit messages from the leader. A leader issues commit messages once it receives
acknowledgments from a quorum of followers. For cross-colo operation, it would be useful to
have a third role: observer. Using Paxos terminology, observers are similar to learners. An
observer does not participate actively in the agreement step of the atomic broadcast protocol.
Instead, it only commits proposals that have been accepted by some quorum of followers.
> One simple solution to implement observers is to have the leader forwarding commit messages
not only to followers but also to observers, and have observers applying transactions according
to the order followers agreed upon. In the current implementation of the protocol, however,
commit messages do not carry their corresponding transaction payload because all servers different
from the leader are followers and followers receive such a payload first through a proposal
message. Just forwarding commit messages as they currently are to an observer consequently
is not sufficient. We have a couple of options:
> 1- Include the transaction payload along in commit messages to observers;
> 2- Send proposals to observers as well.
> Number 2 is simpler to implement because it doesn't require changing the protocol implementation,
but it increases traffic slightly. The performance impact due to such an increase might be
insignificant, though.
> For scalability purposes, we may consider having followers also forwarding commit messages
to observers. With this option, observers can connect to followers, and receive messages from
followers. This choice is important to avoid increasing the load on the leader with the number
of observers. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message