flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Xin <richardxin...@yahoo.com>
Subject Re: how to get topic names in SinkFunction when using FlinkKafkaConsumer010 with multiple topics
Date Thu, 06 Jul 2017 15:39:40 GMT
Thanks,I'm not sure I understand this, what I need is to have single a process subscribing
multiple kafka topics, and have a switch clause for different topics in my SinkFunction,
did you I need to change the way how the kafka producer to produce the message? Any pointer
to code samples will be appreciated. Thanks AgainRichard

On Wednesday, July 5, 2017, 10:25:59 PM PDT, Tzu-Li (Gordon) Tai <tzulitai@apache.org>

#yiv1027114272 body{font-family:Helvetica, Arial;font-size:13px;}Hi Richard,
Producing to multiple topics is treated a bit differently in the Flink Kafka producer.You
need to set a single default target topic, and in `KeyedSerializationSchema#getTargetTopic()`
you can override the default topic with whatever is returned. The `getTargetTopic` method
is invoked for each record.

On 6 July 2017 at 9:09:29 AM, Richard Xin (richardxin168@yahoo.com) wrote:

when usingFlinkKafkaConsumer010 to subscribing multiple topics as 
List<String> topics =Arrays.asList("test1","test2");
DataStream<String> stream = env.addSource(new FlinkKafkaConsumer010<>(topics, new SimpleStringSchema(), properties));
How do I get topic names in mySinkFunction? i.e. stream.addSink()
View raw message