flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4155) Get Kafka producer partition info in open method instead of constructor
Date Wed, 26 Oct 2016 10:25:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15608079#comment-15608079
] 

ASF GitHub Bot commented on FLINK-4155:
---------------------------------------

Github user StephanEwen commented on the issue:

    https://github.com/apache/flink/pull/2681
  
    Looks good to me. I think it would be good if @rmetzger also had a look.
    
    Concerning the Table sink tests and the mocking of the producer: Do these tests need a
Kafka cluster running at all? They are just a think layer on top of the producer, which should
be covered with Unit tests and not need a Kafka cluster.


> Get Kafka producer partition info in open method instead of constructor
> -----------------------------------------------------------------------
>
>                 Key: FLINK-4155
>                 URL: https://issues.apache.org/jira/browse/FLINK-4155
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>    Affects Versions: 1.1.0, 1.0.3
>            Reporter: Gyula Fora
>            Assignee: Tzu-Li (Gordon) Tai
>             Fix For: 1.2.0
>
>
> Currently the Flink Kafka producer does not really do any error handling if something
is wrong with the partition metadata as it is serialized with the user function.
> This means that in some cases the job can go into an error loop when using the checkpoints.
Getting the partition info in the open method would solve this problem (like restarting from
a savepoint which re-runs the constructor).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message