zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Question about the leader-based atomic broadcast
Date Wed, 09 Jan 2019 21:53:56 GMT
All updates to data in Zookeeper are modified to be idempotent before they
are accepted into the leader's queue. That means that items in the queue
can be committed in groups and once each group is acknowledged by a quorum
of servers, it can be deleted from the queue. Any server not in the
acknowledging quorum can get up to date by duplicating the contents of the
leader. As with snapshots, this doesn't require updates to be stopped. The
idempotency of all updates means that the server can duplicate the leader
even as updates are being applied and then can apply all updates that
occurred after the start of the concurrence process. If some of the leader
state was copied before it was affected by an update after the start of
synchronization, it will be updated in the follower's image when replaying
very recent events. If it was copied after being updated, then replaying
recent events will merely write the same value.

The effect of all of this is that there is no penalty for deleting
transactions immediately after they are committed and acknowledged.



On Wed, Jan 9, 2019 at 10:55 AM Meng Xu <xumengpanda@gmail.com> wrote:

> Hi,
>
> I have a question about the leader-based atomic broadcast used in
> ZooKeeper.
>
> According to the Zab paper "A simple totally ordered broadcast
> protocol"[1], the protocol has the requirement of reliable delivery:
> If a message is delivered to one server, it will be eventually
> delivered by all correct servers.
>
> ***My question is:
> When will the leader delete the message from its FIFO queue?
>
> I assume the leader won't delete the message until the message is
> delivered to all servers?
>
> If so, will the leader's FIFO queue keep increasing when there exists
> a slow follower (or slow connection), where it takes a long time for
> the slow follower to receive a message?
>
> Thank you very much!
>
> [1]
> https://www.datadoghq.com/pdf/zab.totally-ordered-broadcast-protocol.2008.pdf
>
> Best,
>
> Meng
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message