flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prez Cannady <revp...@correlatesystems.com>
Subject Re: Attempting to refactor Kafka Example using Flink 0.10.2 with Scala 2.11
Date Wed, 09 Mar 2016 11:42:38 GMT
I’ll give that a shot, but I should also report that as of yesterday I’ve been able to
get it to work with this configuration:

https://github.com/OCExercise/kafka-example/tree/example-2.11-0.9.0.1 <https://github.com/OCExercise/kafka-example/tree/example-2.11-0.9.0.1>

My primary blocking issue seems to have been attempting to publish and subscribe to an older
(~0.8.x.x) Kafka instance.

Prez Cannady  
p: 617 500 3378  
e: revprez@opencorrelate.org <mailto:revprez@opencorrelate.org>  
GH: https://github.com/opencorrelate <https://github.com/opencorrelate>  
LI: https://www.linkedin.com/in/revprez <https://www.linkedin.com/in/revprez>  









> On Mar 9, 2016, at 6:31 AM, Stephan Ewen <sewen@apache.org> wrote:
> 
> Hi!
> 
> Can you use version 1.0.0 for everything? That should make all dependencies consistent.
> 
> Greetings,
> Stephan
> 
> 
> On Wed, Mar 9, 2016 at 11:39 AM, Maximilian Michels <mxm@apache.org <mailto:mxm@apache.org>>
wrote:
> Hi Prez,
> 
> It appears Spring's Classloader is not set up correctly.
> Unfortunately, I'm not familiar with the way Springboot works.
> 
> You added flink-connector-kafka-0.9_2.10 but also mentioned you're
> using Scala 2.11. That is bound to cause troubles :)
> 
> Cheers,
> Max
> 
> On Thu, Mar 3, 2016 at 8:02 PM, Prez Cannady
> <revprez@correlatesystems.com <mailto:revprez@correlatesystems.com>> wrote:
> > I’ve forked and am now experimenting with Robert Metzler’s kaka-example.
> >
> > https://github.com/OCExercise/kafka-example <https://github.com/OCExercise/kafka-example>
> >
> > Work fine from the vanilla fork (on the master branch).  I performed my
> > changes on branch enerscore-2.11, which includes:
> >
> > 1. Going from Flink 0.9 to Flink 0.10.2.
> > 2. Using flink-connector-kafka-0.9_2.10 1.1-SNAPSHOT (which brings in Kafka
> > 0.9).
> > 3. Removed the shade plugin.
> > 4. Using Scala 2.11
> > 5. Instrumenting with spring-boot.
> >
> > Currently running into this problem while attempting to invoke `mvn clean
> > spring-boot:run`:
> >
> > ```
> > [INFO] --- spring-boot-maven-plugin:1.3.1.RELEASE:run (default-cli) @
> > kafka-example ---
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> > [jar:file:/Users/revprez/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> > [jar:file:/Users/revprez/.m2/repository/org/slf4j/slf4j-log4j12/1.7.13/slf4j-log4j12-1.7.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings <http://www.slf4j.org/codes.html#multiple_bindings>
for an
> > explanation.
> > SLF4J: Actual binding is of type
> > [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
> > [WARNING]
> > java.lang.reflect.InvocationTargetException
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >         at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:497)
> >         at
> > org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRunMojo.java:467)
> >         at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.NoClassDefFoundError:
> > org/apache/flink/runtime/state/CheckpointListener
> >         at java.lang.ClassLoader.defineClass1(Native Method)
> >         at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
> >         at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> >         at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> >         at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> >         at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> >         at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >         at java.lang.ClassLoader.defineClass1(Native Method)
> >         at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
> >         at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> >         at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> >         at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> >         at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> >         at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >         at com.dataartisans.ReadFromKafka.main(ReadFromKafka.java:53)
> >         ... 6 more
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.flink.runtime.state.CheckpointListener
> >         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >         ... 29 more
> > ```
> >
> > My guess is that this occurs in the try block of the constructor
> > `FlinkKafkaConsumer09(List<String> topics, KeyedDeserializationSchema<T>
> > deserializer, Properties props)`, and is probably due to me missing
> > something in the classpath.  Any hints on how to proceed would be
> > appreciated.
> >
> >
> > Prez Cannady
> > p: 617 500 3378
> > e: revprez@opencorrelate.org <mailto:revprez@opencorrelate.org>
> > GH: https://github.com/opencorrelate <https://github.com/opencorrelate>
> > LI: https://www.linkedin.com/in/revprez <https://www.linkedin.com/in/revprez>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> 


Mime
View raw message