flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Avoiding duplicates in joined stream
Date Tue, 15 Aug 2017 23:21:56 GMT
What's the best way to avoid duplicates in joined stream. In below code I
get duplicates of "A" because I have multiple of "A" in fileInput3.

SingleOutputStreamOperator<String> fileInput3 = streamEnv.fromElements("A",
"A")

.assignTimestampsAndWatermarks(timestampAndWatermarkAssigner1);

fileInput1.join(fileInput3).where(keySelector1).equalTo(keySelector2)

.window(TumblingEventTimeWindows.*of*(Time.*milliseconds*(3000)))

.apply(function).print();

Mime
View raw message