zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From harish lohar <hklo...@gmail.com>
Subject Fwd: Kafka Failing to start due to existing ID
Date Tue, 12 Jun 2018 20:33:19 GMT
---------- Forwarded message ---------
From: harish lohar <hklohar@gmail.com>
Date: Tue, Jun 12, 2018 at 3:26 PM
Subject: Re: Kafka Failing to start due to existing ID
To: <andor@apache.org>

Hi Andor,

Thanks for your reply.

This issue is irrespective of number of nodes, even should be seen with 3
Node cluster as well.

Actually kafka has session_timeout config , but that seems to be in effect
only if zookeeper cluster is up i.e. if kafka goes down when zookeeper
cluster is up.

Now let's say if 2 nodes of Zookeeper cluster is down , and then if kafka
connected to 3rd Zookeeper Node goes down zookeeper cluster doesn't refresh
the session for Kafka connected to 3rd Node.

So when other Node comes up and zookeeper cluster becomes available it
doesn't delete the id of the kafka which went down when zookeeper cluster
was down.

Regarding TTL I have already enquired the kafka forum and awaiting reply.

Ideally once zookeper cluster is up , it should delete the kafka broker
id's which are not connected which doesn't seem to be happening

I hope I am making some sense :)


On Tue, Jun 12, 2018 at 2:59 PM Andor Molnár <andor@apache.org> wrote:

> Hi Harish,
> I have a few questions to get some insight about your issue.
> 1. Why do run ZooKeeper with 6 nodes while odd number of nodes are
> recommended (not an issue really, just for curiousity),
> 2. Does Kafka support ZK 3.5+ with TTL nodes?
> I think this is more of a Kafka question, but afaik Kafka doesn't run and
> cannot take advantage of 3.5 only features of ZK. Maybe I'm wrong, but I
> think it has some cleanup mechanism to delete expired broker ids or you
> must wait for the session to expire.
> Regards,
> Andor
> On 06/12/2018 04:39 PM, harish lohar wrote:
> Hi All,
> Need help regarding below scenario if any configuration is available to
> help.
> I have cluster of 6 nodes
> 3 Nodes are stopped and  brought up again, kafka fails to restart since
> broker ID are still present in zookeeper znode /broker/ids/
> Since the cluster goes down after removing 3 Nodes , session timeout
> doesn't happen.
> Though i am aware about TTL feature in zookeeper , but how to make sure
> kafka creates znodes with TTL
> Thanks
> Harish

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message