kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: interesting paper on log replication
Date Mon, 15 Apr 2013 04:42:42 GMT
Thanks for the link. This paper provides an alternative, but similar
implementation to that in Zookeeper. The key difference seems to be that
the former supports membership reconfiguration.

Kafka replication is simpler because it separates the leader election part
from log replication. Such separation has a few benefits: (1) the leader
election part is easier to implement by leveraging a consensus system (e.g.
Zookeeper); (2) the log format is simpler since the log itself is not used
for leader election; (3) the replication factor for the log is decoupled
from the number of parties required for leader election (e.g., in Kafka we
can choose a replication factor of 2 for the log while using an ensemble of
5 for a Zookeeper cluster).

Both Rafe and Zookeeper are solving a harder problem than Kafka replication
because they have no consensus service to rely upon for their own leader
election since they are implementing a consensus service.



On Tue, Apr 9, 2013 at 10:34 PM, Jay Kreps <jay.kreps@gmail.com> wrote:

> Very similar in design to kafka replication
> https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf
> -Jay

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message