flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4155) Get Kafka producer partition info in open method instead of constructor
Date Tue, 22 Nov 2016 16:15:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687145#comment-15687145
] 

ASF GitHub Bot commented on FLINK-4155:
---------------------------------------

Github user tzulitai commented on the issue:

    https://github.com/apache/flink/pull/2681
  
    Actually, IIRC, @twalthr already converted the table tests to unit tests in https://issues.apache.org/jira/browse/FLINK-4745.
So, right now the table tests are already unit tests without running a Kafka cluster.
    
    The only change regarding the table tests in this PR is that on the producer side (table
sink), the producer had to be mocked because of the previous partition fetching in the constructor;
this PR merely changes the mocking to directly instantiating.


> Get Kafka producer partition info in open method instead of constructor
> -----------------------------------------------------------------------
>
>                 Key: FLINK-4155
>                 URL: https://issues.apache.org/jira/browse/FLINK-4155
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>    Affects Versions: 1.1.0, 1.0.3
>            Reporter: Gyula Fora
>            Assignee: Tzu-Li (Gordon) Tai
>             Fix For: 1.2.0
>
>
> Currently the Flink Kafka producer does not really do any error handling if something
is wrong with the partition metadata as it is serialized with the user function.
> This means that in some cases the job can go into an error loop when using the checkpoints.
Getting the partition info in the open method would solve this problem (like restarting from
a savepoint which re-runs the constructor).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message