flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tzulitai <...@git.apache.org>
Subject [GitHub] flink pull request #5117: [FLINK-8190] [kafka] Add constructors to expose to...
Date Mon, 04 Dec 2017 11:29:49 GMT
GitHub user tzulitai opened a pull request:


    [FLINK-8190] [kafka] Add constructors to expose topic pattern-based subscription

    ## What is the purpose of the change
    This PR simply exposes functionality that was already implemented.
    There is no new code adding any new functionality.
    Partition and topic regex-pattern-based subscription was implemented in [FLINK-4022](https://issues.apache.org/jira/browse/FLINK-4022).
    However, the functionality for pattern-based topic subscription was never exposed in the
    This PR exposes the feature through the following new constructors on all version-specific
Kafka consumers:
    public FlinkKafkaConsumerXX(java.util.regex.Pattern subscriptionPattern, DeserializationSchema<T>
schema, Properties props);
    public FlinkKafkaConsumerXX(java.util.regex.Pattern subscriptionPattern, KeyedDeserializationSchema<T>
schema, Properties props);
    This allows the consumer to pick up all matching topics on startup.
    To continuously pick up matching topics on the fly when they are created after the job
has already started running, users should additionally set the `KEY_PARTITION_DISCOVERY_INTERVAL_MILLIS`
property, as they would do for partition discovery.
    ## Brief change log
    - Add new constructors for pattern-based topic subscription to `FlinkKafkaConsumer08`,
`..09`, `..010`, and `..011`.
    - Update Kafka connector docs to document the feature.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): no
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
      - The serializers: no
      - The runtime per-record code paths (performance sensitive): no
      - Anything that affects deployment or recovery: no
      - The S3 file system connector: no
    ## Documentation
      - Does this pull request introduce a new feature? yes
      - If yes, how is the feature documented? docs + Javadocs updated

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tzulitai/flink FLINK-8190

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5117
commit 7f7599108c06bbc046b5067bef71dfa859ab01b0
Author: Tzu-Li (Gordon) Tai <tzulitai@apache.org>
Date:   2017-12-04T07:05:46Z

    [FLINK-8190] [kafka] Add constructors to expose topic pattern-based subscription



View raw message