flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8118) Allow to specify the offsets of KafkaTableSources
Date Wed, 22 Nov 2017 09:42:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16262211#comment-16262211

Fabian Hueske commented on FLINK-8118:

Hi [~xccui],
1. I think offsets should be configured via the builder as an optional parameters. IMO, we
should not split the configuration into a builder part and a non-builder part.
2. For testing, you can use Mockito to replace the {{FlinkConsumer}} which is returned from
{{getKafkaConsumer}} by a mock and check if the correct method to configure the offsets is
called. I did something similar to test the configuration of the Orc readers in the [OrcRowInputFormatTest|https://github.com/fhueske/flink/blob/d80506e3785268f541457a69ade3118c634cf7e7/flink-connectors/flink-orc/src/test/java/org/apache/flink/orc/OrcRowInputFormatTest.java].

Best, Fabian

> Allow to specify the offsets of KafkaTableSources
> -------------------------------------------------
>                 Key: FLINK-8118
>                 URL: https://issues.apache.org/jira/browse/FLINK-8118
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>    Affects Versions: 1.4.0
>            Reporter: Timo Walther
>            Assignee: Xingcan Cui
> Right now the Kafka TableSources can only read from the current group offset. We should
expose the possibilities of the Kafka Consumer:
> https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/kafka.html#kafka-consumers-start-position-configuration

This message was sent by Atlassian JIRA

View raw message