flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Scala 2.10/2.11 Maven dependencies
Date Fri, 22 Jan 2016 17:30:29 GMT
+1 for a big notice once we merge this.

I would like to have a suffix-free "flink-streaming-java". However,
I'm having a hard time to refactor the streaming-java code to get rid
of Scala. The streaming API depends on "flink-clients" and
"flink-runtime" which both inherently depend on Scala. Unfortunately,
the streaming API is very much chained with the runtime which makes it
hard to move classes from flink-streaming-java to flink-runtime.

I think the best solution would be to have similar abstraction for the
streaming API as we have in batch. That is, to generate a Plan
independently of the runtime which is later turned into a JobGraph. In
streaming, we have the StreamGraph but it is not runtime independent.
It's a bit of a mess.

Another option I see is to shade away Scala. I did this just to see
how large the binaries would be. It would let "flink-streaming-java"
grow from 3,1 MB (without shading) to 59,7 MB (with shading). That
seems a bit too large for users although it would get rid of the Scala
suffix without refactoring.

On Thu, Jan 21, 2016 at 11:33 PM, Ufuk Celebi <uce@apache.org> wrote:
>> On 21 Jan 2016, at 17:51, Maximilian Michels <mxm@apache.org> wrote:
>> https://issues.apache.org/jira/browse/FLINK-2940
>> There is now a pending pull request: https://github.com/apache/flink/pull/1529
>> As I was working on the changes, I discovered we have some more
>> modules which have a Scala dependency which could be avoided. Namely
>> these are "flink-java8", and "flink-streaming-java". The latter
>> probably affects a lot of users. So it would be nice to make it
>> Scala-free.
>> Do you think that would be feasible?
> Thank you very much for taking care of this. :)
> I would like to have it changed as well before merging for the same reason Fabian brought
up in the PR.
> Since this will be quite a breaking change, we should consider adding a big notice to
the 1.0-SNAPSHOT docs regarding this (basically on the landing page) for the few weeks until
the release.
> – Ufuk

View raw message