kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Toader (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-6598) Kafka to support using ETCD beside Zookeeper
Date Tue, 27 Feb 2018 15:56:00 GMT
Sebastian Toader created KAFKA-6598:

             Summary: Kafka to support using ETCD beside Zookeeper
                 Key: KAFKA-6598
                 URL: https://issues.apache.org/jira/browse/KAFKA-6598
             Project: Kafka
          Issue Type: New Feature
          Components: clients, core
            Reporter: Sebastian Toader

The current Kafka implementation is bound to {{Zookeeper}} to store its metadata for forming
a cluster of nodes (producer/consumer/broker). 
As Kafka is becoming popular for streaming in various environments where {{Zookeeper}} is
either not easy to deploy/manage or there are better alternatives to it there is a need 
to run Kafka with other metastore implementation than {{Zookeeper}}.

{{etcd}} can provide the same semantics as {{Zookeeper}} for Kafka and since {{etcd}} is the
favorable choice in certain environments (e.g. Kubernetes) Kafka should be able to run with
>From the user's point of view should be straightforward to configure to use {{etcd}} by
just simply specifying a connection string that point to {{etcd}} cluster.

To avoid introducing instability the original interfaces should be kept and only the low level
{{Zookeeper}} API calls should be replaced with \{{etcd}} API calls in case Kafka is configured

to use {{etcd}}.

On the long run (which is out of scope of this jira) there should be an abstract layer in
Kafka which then various metastore implementations would implement.

This message was sent by Atlassian JIRA

View raw message