flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: using regular expression to specify Kafka topics
Date Mon, 04 Dec 2017 07:02:18 GMT
Hi Soheil,

That feature is actually already internally available.
The only issue is that the functionality is not yet exposed via any public APIs on the Kafka
Please see this JIRA here: https://issues.apache.org/jira/browse/FLINK-8190.

I’m not sure of exposing the pattern-based subscription API now for 1.4, though.
Although it is only a matter of exposing already implemented features, AFAIK pattern-based
subscription is not thoroughly tested in the release candidates.
cc @Aljoscha, what do you think about this?


On 3 December 2017 at 5:29:32 AM, Soheil Pourbafrani (soheil.ir08@gmail.com) wrote:

I use Flink Kafka connector 10 to subscribe topics and get data. Now I want to specify topics
not using String, but regular expression. I want to do that just because it can recognize
future topics added to the Kafka and get their data.

Spark Kafka connector has a method named SubscribePattern and we can use it to subscribe to
topics using regular expressions.

Is there something equivalent or some other way to do that?
View raw message