flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: using regular expression to specify Kafka topics
Date Mon, 04 Dec 2017 11:36:38 GMT

I’ve created a PR to publicly expose the feature: https://github.com/apache/flink/pull/5117.
Whether or not we should include this in the next release candidate for 1.4 is still up for

On 4 December 2017 at 3:02:29 PM, Tzu-Li (Gordon) Tai (tzulitai@apache.org) wrote:

Hi Soheil,

That feature is actually already internally available.
The only issue is that the functionality is not yet exposed via any public APIs on the Kafka
Please see this JIRA here: https://issues.apache.org/jira/browse/FLINK-8190.

I’m not sure of exposing the pattern-based subscription API now for 1.4, though.
Although it is only a matter of exposing already implemented features, AFAIK pattern-based
subscription is not thoroughly tested in the release candidates.
cc @Aljoscha, what do you think about this?


On 3 December 2017 at 5:29:32 AM, Soheil Pourbafrani (soheil.ir08@gmail.com) wrote:

I use Flink Kafka connector 10 to subscribe topics and get data. Now I want to specify topics
not using String, but regular expression. I want to do that just because it can recognize
future topics added to the Kafka and get their data.

Spark Kafka connector has a method named SubscribePattern and we can use it to subscribe to
topics using regular expressions.

Is there something equivalent or some other way to do that?
View raw message