flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject [DISCUSS] API breaking change in DataStream Windows
Date Mon, 08 Aug 2016 14:11:23 GMT
Hi all!

We have a problem in the *DataStream API* around Windows for *CoGroup* and
*Join*.
These operations currently do not allow to set a parallelism, which is a
pretty heavy problem.

To fix it properly, we need to change the return types of the coGroup() and
join() operations, which *breaks the binary compatibility* - it* retains
source compatibility*, though.

The pull request with the change is:
https://github.com/apache/flink/pull/2305

There are very clumsy ways to work around this (custom casts in the user
code or making the join() / coGroup() behave differently than the other
operators) which we did not really think of as viable, because they would
need to be changed again in the future once we pull the API straight
(breaking even source compatibility then).

*I would suggest to actually break the API* at that point (binary, not
source) for *Flink 1.2* and add a big note in the release docs. An
uncomfortable step, but the alternatives are quite bad, too.

Have a look at what has been suggested in the pull request discussion and
please let us know what you think about that so we can proceed.

Greetings,
Stephan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message