flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@apache.org>
Subject Re: flink-dist packaging including unshaded classes
Date Wed, 09 Dec 2015 23:35:22 GMT
Thanks, I appreciate it.

On Wed, Dec 9, 2015 at 12:50 PM, Robert Metzger <rmetzger@apache.org> wrote:

> I can confirm that guava is part of the fat jar for the 2.7.0, scala 2.11
> distribution.
>
> I'll look into the issue tomorrow
>
> On Wed, Dec 9, 2015 at 7:58 PM, Nick Dimiduk <ndimiduk@apache.org> wrote:
>
> > mvn dependency:tree from flink-dist module does not include any mention
> of
> > guava. When I build (mvn clean package -DskipTests) vs master (fc8be1c) I
> > see the same packaging problem.
> >
> > On Wed, Dec 9, 2015 at 9:29 AM, Stephan Ewen <sewen@apache.org> wrote:
> >
> > > Usually, no command line magic is needed. Simple "mvn clean package"
> does
> > > it.
> > >
> > > May be that this is not related to your building at all. Can you check
> > > whether the same is already the case on the master branch with Hadoop
> 2.7
> > > Scala 2.11 ?
> > >
> > > Also, simple way to diagnose where these dependencies come from is to
> do
> > > inside the "flink-dist" project a "mvn dependency:tree" run. That shows
> > how
> > > the unshaded Guava was pulled in.
> > >
> > > Greetings,
> > > Stephan
> > >
> > >
> > > On Wed, Dec 9, 2015 at 6:22 PM, Nick Dimiduk <ndimiduk@gmail.com>
> wrote:
> > >
> > > > I did not. All I did was apply the PR from FLINK-3147. I thought
> > perhaps
> > > > there's some command line incantation I'm missing.
> > > >
> > > > On Wed, Dec 9, 2015 at 3:29 AM, Stephan Ewen <sewen@apache.org>
> wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > > Did you change anything in the POM files, with respect to Guava,
or
> > add
> > > > > another dependency that might transitively pull Guava?
> > > > >
> > > > > Stephan
> > > > >
> > > > >
> > > > > On Tue, Dec 8, 2015 at 9:25 PM, Nick Dimiduk <ndimiduk@apache.org>
> > > > wrote:
> > > > >
> > > > > > Hi there,
> > > > > >
> > > > > > I'm attempting to build locally a flink based on release-0.10.0
+
> > > > > > FLINK-3147. When I build from this sandbox, the resulting
> > > > flink-dist.jar
> > > > > > contains both shanded and unshaded jars. In my case, this results
> > in
> > > a
> > > > > > runtime conflict in my application, where
> > > > > com.google.common.base.Stopwatch
> > > > > > from both Guava-12 and Guava-18 are in my classpath.
> > > > > >
> > > > > > Is there some additional profile required to build a dist package
> > > with
> > > > > only
> > > > > > the shaded jars?
> > > > > >
> > > > > > Thanks,
> > > > > > Nick
> > > > > >
> > > > > > $ tar xvzf flink-0.10.0-bin-hadoop27-scala_2.11.tgz
> > > > > > $ cd flink-0.10.0
> > > > > > $ $ unzip -t lib/flink-dist_2.11-0.10.0.jar | grep Stopwatch
> > > > > >     testing:
> > > > > > org/apache/flink/shaded/com/google/common/base/Stopwatch$1.class
> >  OK
> > > > > >     testing:
> > > > > org/apache/flink/shaded/com/google/common/base/Stopwatch.class
> > > > > >   OK
> > > > > >     testing:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org/apache/flink/shaded/com/google/common/util/concurrent/RateLimiter$SleepingStopwatch$1.class
> > > > > >   OK
> > > > > >     testing:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org/apache/flink/shaded/com/google/common/util/concurrent/RateLimiter$SleepingStopwatch.class
> > > > > >   OK
> > > > > >     testing:
> > > > > >
> > > org/apache/flink/hadoop/shaded/com/google/common/base/Stopwatch$1.class
> > > > > >  OK
> > > > > >     testing:
> > > > > >
> > org/apache/flink/hadoop/shaded/com/google/common/base/Stopwatch.class
> > > > >  OK
> > > > > >     testing: com/google/inject/internal/util/$Stopwatch.class
>  OK
> > > > > >
> > > > > > vs.
> > > > > >
> > > > > > $ git status
> > > > > > HEAD detached from release-0.10.0
> > > > > > $ git log --decorate=short --oneline | head -n3
> > > > > > dccdbd8 (HEAD) [FLINK-3147] HadoopOutputFormatBase should expose
> > > fields
> > > > > as
> > > > > > protected
> > > > > > ab2cca4 (tag: release-0.10.0, origin/release-0.10.0-rc8,
> > > > > > ndimiduk/release-0.10.0-rc8) Commit for release 0.10.0
> > > > > > c0fe305 [FLINK-2992] Remove use of SerializationUtils
> > > > > > $ mvn clean install -DskipTests
> > > > > > ...
> > > > > > $ cd flink-dist/target/flink-0.10.0-bin/flink-0.10.0
> > > > > > $ unzip -t lib/flink-dist-0.10.0.jar | grep Stopwatch
> > > > > >     testing:
> > > > > > org/apache/flink/shaded/com/google/common/base/Stopwatch$1.class
> >  OK
> > > > > >     testing:
> > > > > org/apache/flink/shaded/com/google/common/base/Stopwatch.class
> > > > > >   OK
> > > > > >     testing:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org/apache/flink/shaded/com/google/common/util/concurrent/RateLimiter$SleepingStopwatch$1.class
> > > > > >   OK
> > > > > >     testing:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org/apache/flink/shaded/com/google/common/util/concurrent/RateLimiter$SleepingStopwatch.class
> > > > > >   OK
> > > > > >     testing:
> > > > > >
> > > org/apache/flink/hadoop/shaded/com/google/common/base/Stopwatch$1.class
> > > > > >  OK
> > > > > >     testing:
> > > > > >
> > org/apache/flink/hadoop/shaded/com/google/common/base/Stopwatch.class
> > > > >  OK
> > > > > >     testing: com/google/inject/internal/util/$Stopwatch.class
>  OK
> > > > > >     testing: com/google/common/base/Stopwatch$1.class   OK
> > > > > >     testing: com/google/common/base/Stopwatch.class   OK
> > > > > >     testing:
> > > > > >
> > > com/google/common/util/concurrent/RateLimiter$SleepingStopwatch$1.class
> > > > > >  OK
> > > > > >     testing:
> > > > > >
> > com/google/common/util/concurrent/RateLimiter$SleepingStopwatch.class
> > > > >  OK
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message