flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Eclipse issue after latest dependency rework pr
Date Wed, 01 Oct 2014 15:43:20 GMT
I've just updated the streaming fork with Stefan's recent commit
encapsulating this issue, namely:
https://github.com/mbalassi/incubator-flink/commit/949699dbfe17b62352413769635aed3aaff56100

It solves the problem for the streaming-core project, but running the batch
WordCount example in Eclipse still produces the following:

Executing WordCount example with built-in default data.
  Provide parameters to read input data from a file.
  Usage: WordCount <text path> <result path>
Exception in thread "main" java.lang.NoClassDefFoundError:
com/google/common/base/Preconditions
at
org.apache.flink.api.common.operators.util.UserCodeObjectWrapper.<init>(UserCodeObjectWrapper.java:40)
at
org.apache.flink.api.common.operators.base.GenericDataSourceBase.<init>(GenericDataSourceBase.java:58)
at
org.apache.flink.api.java.operators.DataSource.translateToDataFlow(DataSource.java:75)
at
org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:82)
at
org.apache.flink.api.java.operators.OperatorTranslation.translateSingleInputOperator(OperatorTranslation.java:117)
at
org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:85)
at
org.apache.flink.api.java.operators.OperatorTranslation.translateSingleInputOperator(OperatorTranslation.java:117)
at
org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:85)
at
org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:60)
at
org.apache.flink.api.java.operators.OperatorTranslation.translateToPlan(OperatorTranslation.java:48)
at
org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:650)
at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:48)
at
org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:82)
Caused by: java.lang.ClassNotFoundException:
com.google.common.base.Preconditions
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more

Of course one approach is to move

<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
<scope>provided</scope>
</dependency>

towards the flink-parent pom, but this ultimately works against shading and
the hadoop 2.5 version is going to fail as it does here:

https://travis-ci.org/mbalassi/incubator-flink/jobs/36780802

Any suggestions? :)

On Wed, Oct 1, 2014 at 9:40 AM, Gyula Fora <gyfora@apache.org> wrote:

> That worked, thanks.
>
> On 30 Sep 2014, at 23:12, Stephan Ewen <sewen@apache.org> wrote:
>
> > Yes, I wanted to write a few pointers about such issues.
> >
> > We recently shaded the guava dependency, meaning that we have a custom
> > version of guava where all classes reside in
> > "org.apache.flink.shaded.com.google" and maven rewrites all our guava
> > references to that namespace.
> >
> > The reason is that guava is a frequently used library and versions are
> not
> > all compatible. For example, hadoop 2.5 uses guava 11 and fails if we
> bring
> > guava 17 into the classpath. The shading causes our guava version to not
> > conflict, because our shaded version's classes reside different
> namespace.
> >
> > Since Eclipse does not interpret the shading (it only happens in the
> maven
> > package phase) you need a regular dependency to guava that is not
> exported
> > ("provided" scope)
> >
> > Try adding this to the project's pom, it will tell maven to compile and
> > individually run with the guava lib, but not package it.
> >
> > <dependency>
> > <groupId>com.google.guava</groupId>
> > <artifactId>guava</artifactId>
> > <version>${guava.version}</version>
> > <scope>provided</scope>
> > </dependency>
> >
> >
> >
> > On Tue, Sep 30, 2014 at 11:02 PM, Gyula Fóra <gyfora@apache.org> wrote:
> >
> >> Hey,
> >>
> >> We have pulled the dependency rework from the apache master and now we
> >> cannot get our tests to run in eclipse. With maven from the command line
> >> and also with travis it works perfectly but when I try to run tests that
> >> access the the Configuration class we get the following exception:
> >>
> >> java.lang.NoClassDefFoundError: com/google/common/io/BaseEncoding
> >> at
> >>
> >>
> org.apache.flink.configuration.Configuration.setBytes(Configuration.java:358)
> >> at
> >> .....
> >> some stuff here
> >> .....
> >>
> >> Caused by: java.lang.ClassNotFoundException:
> >> com.google.common.io.BaseEncoding
> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> >> ... 32 more
> >>
> >> I literally tried everything to make it work, deleted all maven files,
> >> reinstalled eclipse etc, but still no luck. Do you experience the same
> >> issue when running the streaming-core tests in eclipse?
> >>
> >> Regards,
> >> Gyula
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message