flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Gryzlov <alex.gryz...@gmail.com>
Subject synchronizing two streams
Date Wed, 11 May 2016 15:00:31 GMT

We're implementing a streaming outer join operator based on a
TwoInputStreamOperator with an internal buffer. In our use-case only the
items whose timestamps are within a several-second interval of each other
can join, so we need to synchronize the two input streams to ensure maximal
yield. Our plan is to utilize the watermark mechanism to implement some
sort of a "throttling" operator, which would take two streams and stop
passing through one of them based on the watermarks in another. However,
there doesn't seem to exist an operator of the shape (A,B)->(A,B) in Flink,
where A and B can be received and emitted independently. What would be a
resource-saving way to implement such (e.g., without spawning two more
parallel TwoInputStreamOperators)?


View raw message