kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srinivas, Kaushik (Nokia - IN/Bangalore)" <kaushik.srini...@nokia.com>
Subject RE: kafka connect stops consuming data when kafka broker goes down
Date Fri, 07 Jun 2019 07:43:26 GMT
Hi Paul,

We tried patching out kafka connect as well referencing the PR which you had shared. But no
improvement in our case.
Have updated more details in the blocker issue which I have created for the same.
https://issues.apache.org/jira/browse/KAFKA-8485

But the issue seems to be coming from the internal producers/consumers getting into hung state
after the broker restarts.

Any other lead/inputs would be useful.
This issue is always reproducible even with/without data streaming in progress.

Thanks,
Kaushik

-----Original Message-----
From: Paul Whalen <pgwhalen@gmail.com> 
Sent: Wednesday, June 05, 2019 5:17 PM
To: dev@kafka.apache.org
Cc: Basil Brito, Aldan (Nokia - IN/Bangalore) <aldan.basil_brito@nokia.com>
Subject: Re: kafka connect stops consuming data when kafka broker goes down

It’s not totally clear, but this may be https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-7941

For which there is a fix that is very nearly approved: https://github.com/apache/kafka/pull/6283

Paul

> On Jun 5, 2019, at 1:26 AM, Srinivas, Kaushik (Nokia - IN/Bangalore) <kaushik.srinivas@nokia.com>
wrote:
> 
> Hello,
> Anyone has any information on this issue.
> Created a critical ticket for the same, since this is a major stability issue for connect
framework.
> https://issues.apache.org/jira/browse/KAFKA-8485?filter=-2
> 
> Thanks.
> Kaushik,
> NOKIA
> 
> From: Srinivas, Kaushik (Nokia - IN/Bangalore)
> Sent: Monday, June 03, 2019 5:22 PM
> To: dev@kafka.apache.org
> Cc: Basil Brito, Aldan (Nokia - IN/Bangalore) 
> <aldan.basil_brito@nokia.com>
> Subject: kafka connect stops consuming data when kafka broker goes 
> down
> 
> Hello kafka dev,
> 
> We are encountering an issue when kafka connect is running hdfs sink connector pulling
data from kafka and writing to hdfs location.
> In between when the data is flowing in the pipeline from producer -> 
> kafka topic -> kafka connect hdfs sink connector -> hdfs, If even one of the kafka
broker goes down, the connect framework stops responding. Stops consuming records and REST
API also becomes not interactive.
> 
> Until the kafka connect framework is restarted, it would not pull the data from kafka
and REST api remains inactive. Nothing is coming in the logs as well.
> Checked the topics in kafka used by connect, everything has been reassigned to another
broker and has the leader available.
> 
> Has anyone encountered this issue ? what would be the expected behavior ?
> 
> Thanks in advance
> Kaushik
Mime
View raw message