flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Scala 2.10/2.11 Maven dependencies
Date Mon, 25 Jan 2016 10:39:38 GMT
I won't have the time to finish the refactoring. Also, it will be
pretty painful with all the large streaming pull requests being merged
at the moment. If there are no objections, I would like to merge the
Scala suffix changes with "flink-streaming-java" being Scala
dependent. It will improve the experience for users in the long run.

After merging, I'll announce the new Scala suffixed modules. In
addition, we could deploy an empty Maven module to overwrite the
current snapshot version of flink-streaming-java. That would prevent
conflicts with different snapshot versions, e.g. combining
flink-streaming-java (instead of flink-streaming-java_2.11) with
flink-runtime_2.11. Once 1.0.0 is out, the old artifacts won't be a
problem anymore.

Any objections?

On Fri, Jan 22, 2016 at 6:30 PM, Maximilian Michels <mxm@apache.org> wrote:
> +1 for a big notice once we merge this.
>
> I would like to have a suffix-free "flink-streaming-java". However,
> I'm having a hard time to refactor the streaming-java code to get rid
> of Scala. The streaming API depends on "flink-clients" and
> "flink-runtime" which both inherently depend on Scala. Unfortunately,
> the streaming API is very much chained with the runtime which makes it
> hard to move classes from flink-streaming-java to flink-runtime.
>
> I think the best solution would be to have similar abstraction for the
> streaming API as we have in batch. That is, to generate a Plan
> independently of the runtime which is later turned into a JobGraph. In
> streaming, we have the StreamGraph but it is not runtime independent.
> It's a bit of a mess.
>
> Another option I see is to shade away Scala. I did this just to see
> how large the binaries would be. It would let "flink-streaming-java"
> grow from 3,1 MB (without shading) to 59,7 MB (with shading). That
> seems a bit too large for users although it would get rid of the Scala
> suffix without refactoring.
>
> On Thu, Jan 21, 2016 at 11:33 PM, Ufuk Celebi <uce@apache.org> wrote:
>>
>>> On 21 Jan 2016, at 17:51, Maximilian Michels <mxm@apache.org> wrote:
>>>
>>> https://issues.apache.org/jira/browse/FLINK-2940
>>>
>>> There is now a pending pull request: https://github.com/apache/flink/pull/1529
>>>
>>> As I was working on the changes, I discovered we have some more
>>> modules which have a Scala dependency which could be avoided. Namely
>>> these are "flink-java8", and "flink-streaming-java". The latter
>>> probably affects a lot of users. So it would be nice to make it
>>> Scala-free.
>>>
>>> Do you think that would be feasible?
>>
>> Thank you very much for taking care of this. :)
>>
>> I would like to have it changed as well before merging for the same reason Fabian
brought up in the PR.
>>
>> Since this will be quite a breaking change, we should consider adding a big notice
to the 1.0-SNAPSHOT docs regarding this (basically on the landing page) for the few weeks
until the release.
>>
>> – Ufuk
>>

Mime
View raw message