flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-6529) Rework the shading model in Flink
Date Wed, 10 May 2017 16:02:04 GMT
Stephan Ewen created FLINK-6529:

             Summary: Rework the shading model in Flink
                 Key: FLINK-6529
                 URL: https://issues.apache.org/jira/browse/FLINK-6529
             Project: Flink
          Issue Type: Bug
          Components: Build System
    Affects Versions: 1.2.1, 1.3.0
            Reporter: Stephan Ewen
            Priority: Critical

h2. Problem

Currently, Flink shades dependencies like ASM and Guava into all jars of projects that reference
it and relocate the classes.

There are some drawbacks to that approach, let's discuss them at the example of ASM:

  - The ASM classes are for example in {{flink-core}}, {{flink-java}}, {{flink-scala}}, {{flink-runtime}},

  - Users that reference these dependencies have the classes multiple times in the classpath.
That is unclean (works, through, because the classes are identical). The same happens when
building the final dist. jar.

  - Some of these dependencies require to include license files in the shaded jar. It is hard
to impossible to build a good automatic solution for that, partly due to Maven's very poor
cross-project path support

  - Scala does not support shading really well. Scala classes have references to classes in
more places than just the class names (apparently for Scala reflect support). Referencing
a Scala project with shaded ASM still requires to add a reference to unshaded ASM (at least
as a compile dependency).

h2. Proposal

I propose that we build and deploy a {{asm-flink-shaded}} version of ASM and directly program
against the relocated namespaces. Since we never use classes that we relocate in public interfaces,
Flink users will never see the relocated class names. Internally, it does not hurt to use

  - Proper maven dependency management, no hidden (shaded) dependencies

  - one copy of each dependency

  - proper Scala interoperability

  - no clumsy license management (license is in the deployed {{asm-flink-shaded}})

This message was sent by Atlassian JIRA

View raw message