flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasiliki Kalavri <vasilikikala...@gmail.com>
Subject Union a data stream with a product of itself
Date Tue, 24 Nov 2015 15:46:52 GMT
Hi squirrels,

when porting the gelly streaming code from 0.9 to 0.10 today with Paris, we
hit an exception in union: "*A DataStream cannot be unioned with itself*".

The code raising this exception looks like this:
stream.union(stream.map(...)).

Taking a look into the union code, we see that it's now not allowed to
union a stream, not only with itself, but with any product of itself.

First, we are wondering, why is that? Does it make building the stream
graph easier in some way?
Second, we might want to give a better error message there, e.g. "*A
DataStream cannot be unioned with itself or a product of itself*", and
finally, we should update the docs, which currently state that union a
stream with itself is allowed and that "*If you union a data stream with
itself you will still only get each element once.*"

Cheers,
-Vasia.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message