flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: dynamic add sink to flink
Date Mon, 12 Jun 2017 15:30:59 GMT

The Flink Kafka Producer allows writing to multiple topics beside the default topic.
To do this, you can override the configured default topic by implementing the `getTargetTopic`
method on the `KeyedSerializationSchema`.
That method is invoked for each record, and if a value is returned, the record will be routed
to that topic instead of the default one.

Does this address what you have in mind?


On 10 June 2017 at 6:20:59 AM, qq (468137306@qq.com) wrote:

we use flink as a router to our kafka, we read from one kafka and to a lot of diffrent kafka
and topic, 
but it will cost too much time to start the flink job if we want to add some other kafka sink
to the flink, so 
if there any way to dynamic add sink to flink or just start the flink job faster? The reason
of slow start flink 
job I think is the lots of kafka sink. 

We now use the demo code like this, 

splitStream = stream.split(by the content) 
for ((k, v) <- map) { 
splitStream.select(k).addSink(new kafkaSink(v)) 

View raw message