flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Joireman <paul.joire...@physiq.com>
Subject Re: Getting key from keyed stream
Date Thu, 12 Jan 2017 23:19:04 GMT
Thanks Jamie,

Just figured that out after some digging and a little trial and error, that works great.


From: Jamie Grier <jamie@data-artisans.com>
Sent: Thursday, January 12, 2017 4:59:43 PM
To: user@flink.apache.org
Subject: Re: Getting key from keyed stream

A simpler and more efficient approach would simply be the following:

val stream = env.addSource(new FlinkKafkaConsumer(...))

  .addSink(new FlinkKafkaProducer(new MyKeyedSerializationSchema(...)))


In MyKeyedSerializationSchema just override the getTargetTopic() method.

That should do it :)


On Thu, Jan 12, 2017 at 12:53 PM, Paul Joireman <paul.joireman@physiq.com<mailto:paul.joireman@physiq.com>>

Hi all,

Is there a simple way to read the key from a KeyedStream.   Very simply I'm trying to read
a message from Kafka, separate the incoming messages by a field in the message and write the
original message back to Kafka using that field as a new topic.  I chose to partition the
incoming stream by creating a KeyedStream and using the field from the message as the key.
   The only thing left is to write the message to Kafka with a producer but i need to know
the topic to write to and for that I need to be able to read the key.   Is there a way to
do this?

Is there a better way to do this, rather than using a KeyedStream.



Jamie Grier
data Artisans, Director of Applications Engineering

View raw message