flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-8190) Add extra FlinkKafkaConsumer constructors to expose pattern-based topic subscription functionality
Date Mon, 04 Dec 2017 06:27:00 GMT
Tzu-Li (Gordon) Tai created FLINK-8190:
------------------------------------------

             Summary: Add extra FlinkKafkaConsumer constructors to expose pattern-based topic
subscription functionality
                 Key: FLINK-8190
                 URL: https://issues.apache.org/jira/browse/FLINK-8190
             Project: Flink
          Issue Type: New Feature
          Components: Kafka Connector
            Reporter: Tzu-Li (Gordon) Tai
            Assignee: Tzu-Li (Gordon) Tai
             Fix For: 1.4.0, 1.5.0


The required internals for pattern-based topic discovery was implemented as part of FLINK-6022
(along with partition discovery). However, the functionality for pattern-based topic discovery
was not yet exposed via any visible constructor on the version-specific subclasses of {{FlinkKafkaConsumerBase}}.

I propose to add two more constructors for this:
{code}
public FlinkKafkaConsumerXX(Pattern subscriptionPattern, DeserializationSchema<T> schema,
Properties props);

public FlinkKafkaConsumerXX(Pattern subscriptionPattern, KeyedDeserializationSchema<T>
schema, Properties props);
{code}

This allows the consumer to pick up all matching topics on startup.
To continuously pick up matching topics on the fly when they are created after the job has
already started running, users should additionally set the {{KEY_PARTITION_DISCOVERY_INTERVAL_MILLIS}}
property, as they would do for partition discovery.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message