kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kaushik srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-8485) Kafka connect worker does not respond when kafka broker goes down with data streaming in progress
Date Wed, 05 Jun 2019 06:01:00 GMT
kaushik srinivas created KAFKA-8485:
---------------------------------------

             Summary: Kafka connect worker does not respond when kafka broker goes down with
data streaming in progress
                 Key: KAFKA-8485
                 URL: https://issues.apache.org/jira/browse/KAFKA-8485
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
    Affects Versions: 2.2.1
            Reporter: kaushik srinivas


Below is the scenario

3 kafka brokers are up and running.

Kafka connect worker is installed and a hdfs sink connector is added.

Data streaming started, data being flushed out of kafka into hdfs.

Topic is created with 3 partitons, one leader on all the three brokers.

Now, 2 kafka brokers are restarted. Partition re balance happens.

Now we observe, kafka connect does not respond. REST API keeps timing out. 

Nothing useful is being logged at the connect logs as well.

Only way to get out of this situation currently is to restart the kafka connect worker and
things gets normal.

 

The same scenario when tried without data being in progress, works fine. Meaning REST API
does not get into timing out state. 

making this issue a blocker, because of the impact due to kafka broker restart.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message