flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selvaraj chennappan <selvarajchennap...@gmail.com>
Subject connecting two streams flink
Date Tue, 29 Jan 2019 12:43:52 GMT

I have two kafka consumer for same topic and want to join second stream to
first after couple of subtasks computation in the first stream then
validate the record . KT - C1 ,C2

KT - C1 - Transformation(FlatMap) - Dedup - Validate --ifvalidsave it to DB
-C2 - Process --------------------------

if record is invalid then save it to Error topic .

How do I merge these two streams one(c2) is fast and other(c1) is little
slow (two levels of computation) ?
Same record is flowing from C1-Flatmap-FlatMap and other consumer C2 . I
have to validate that record based on the rules.
Please find the attached image herewith reference.
[image: two-stream.png]


Selvaraj C

View raw message