Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 27C9519AAF for ; Wed, 9 Mar 2016 11:42:50 +0000 (UTC) Received: (qmail 42781 invoked by uid 500); 9 Mar 2016 11:42:49 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 42694 invoked by uid 500); 9 Mar 2016 11:42:49 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 42682 invoked by uid 99); 9 Mar 2016 11:42:49 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Mar 2016 11:42:49 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 2D39CC035F for ; Wed, 9 Mar 2016 11:42:49 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.88 X-Spam-Level: * X-Spam-Status: No, score=1.88 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=correlatesystems.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 10qGtcDSH1r5 for ; Wed, 9 Mar 2016 11:42:46 +0000 (UTC) Received: from mail-yw0-f180.google.com (mail-yw0-f180.google.com [209.85.161.180]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id E808F5FAD6 for ; Wed, 9 Mar 2016 11:42:45 +0000 (UTC) Received: by mail-yw0-f180.google.com with SMTP id h129so37199844ywb.1 for ; Wed, 09 Mar 2016 03:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=correlatesystems.com; s=google; h=from:message-id:mime-version:subject:date:references:to:in-reply-to; bh=ysB+4gh95KD05SLFlwXnIjNpHl3t3jy+bOQgMsSB17U=; b=JdWz4cWt4dj/KZgckds3s/63QYAdpEkDls3I5USIV2V/H3G63F53OD/0BCUR2gZQj/ 6JKevNS8+VF7uUk4jRNjRJX9/wP+Xrt/zS9asZJKXMG6rWfhhAdkYbDJtsUJ9Kf5HzVI jgQaKNH/feSvkZqZj5LKX86ZPnFDjDvHOX/f4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:message-id:mime-version:subject:date :references:to:in-reply-to; bh=ysB+4gh95KD05SLFlwXnIjNpHl3t3jy+bOQgMsSB17U=; b=T96PkFDhJedEQlwaN0MahvIOWr2nqA2ATvsXJAcBbgUdSLj/uaDQ+arakMxjxlzqaa h4PfUOmOh1C7hzeBvlbAzxfCMn8F8YwDUbgaLA2GNv3cI7Zi4l+TCmv6+BjyFKVp5R0N n3YUpvZNs0/9esZIC2unMMU3+oXGgiEu0K4huwcGyhP8Y0bcD3Tty9y60d/c2seVCoC1 YW2kkJhPfc4bXEQTWRIuxnnEY0nrtJ+WtKOICAPkcQNSKbnUzIon4NxOwQ3qdnogMAbf CC16owobfM/1rAS+4efOOcyw8D1aD9dhDVtEDAa1AOQeeD+t42jX+2lCr+a2oqrjeVyK ZiHQ== X-Gm-Message-State: AD7BkJIiiSST20SlYJZOjOH+eh7LcZq2IpVEfEGS/mtMSR2c5rH39RolW6AhtrfL1hOJWw== X-Received: by 10.13.254.67 with SMTP id o64mr17813080ywf.77.1457523765195; Wed, 09 Mar 2016 03:42:45 -0800 (PST) Received: from [10.0.40.101] (static-100-0-198-149.bstnma.fios.verizon.net. [100.0.198.149]) by smtp.gmail.com with ESMTPSA id i6sm4566153ywe.11.2016.03.09.03.42.44 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 09 Mar 2016 03:42:44 -0800 (PST) From: Prez Cannady Content-Type: multipart/alternative; boundary="Apple-Mail=_1D1B4C03-213B-4B44-BEDE-5C4DB7C7C7E8" Message-Id: <824194D6-F043-4139-8191-48284159BA14@correlatesystems.com> Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Attempting to refactor Kafka Example using Flink 0.10.2 with Scala 2.11 Date: Wed, 9 Mar 2016 06:42:38 -0500 References: To: user@flink.apache.org In-Reply-To: X-Mailer: Apple Mail (2.3112) --Apple-Mail=_1D1B4C03-213B-4B44-BEDE-5C4DB7C7C7E8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I=E2=80=99ll give that a shot, but I should also report that as of = yesterday I=E2=80=99ve been able to get it to work with this = configuration: https://github.com/OCExercise/kafka-example/tree/example-2.11-0.9.0.1 = My primary blocking issue seems to have been attempting to publish and = subscribe to an older (~0.8.x.x) Kafka instance. Prez Cannady =20 p: 617 500 3378 =20 e: revprez@opencorrelate.org =20 GH: https://github.com/opencorrelate =20= LI: https://www.linkedin.com/in/revprez = =20 > On Mar 9, 2016, at 6:31 AM, Stephan Ewen wrote: >=20 > Hi! >=20 > Can you use version 1.0.0 for everything? That should make all = dependencies consistent. >=20 > Greetings, > Stephan >=20 >=20 > On Wed, Mar 9, 2016 at 11:39 AM, Maximilian Michels > wrote: > Hi Prez, >=20 > It appears Spring's Classloader is not set up correctly. > Unfortunately, I'm not familiar with the way Springboot works. >=20 > You added flink-connector-kafka-0.9_2.10 but also mentioned you're > using Scala 2.11. That is bound to cause troubles :) >=20 > Cheers, > Max >=20 > On Thu, Mar 3, 2016 at 8:02 PM, Prez Cannady > > = wrote: > > I=E2=80=99ve forked and am now experimenting with Robert Metzler=E2=80= =99s kaka-example. > > > > https://github.com/OCExercise/kafka-example = > > > > Work fine from the vanilla fork (on the master branch). I performed = my > > changes on branch enerscore-2.11, which includes: > > > > 1. Going from Flink 0.9 to Flink 0.10.2. > > 2. Using flink-connector-kafka-0.9_2.10 1.1-SNAPSHOT (which brings = in Kafka > > 0.9). > > 3. Removed the shade plugin. > > 4. Using Scala 2.11 > > 5. Instrumenting with spring-boot. > > > > Currently running into this problem while attempting to invoke `mvn = clean > > spring-boot:run`: > > > > ``` > > [INFO] --- spring-boot-maven-plugin:1.3.1.RELEASE:run (default-cli) = @ > > kafka-example --- > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in > > = [jar:file:/Users/revprez/.m2/repository/ch/qos/logback/logback-classic/1.1= .3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: Found binding in > > = [jar:file:/Users/revprez/.m2/repository/org/slf4j/slf4j-log4j12/1.7.13/slf= 4j-log4j12-1.7.13.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings = for an > > explanation. > > SLF4J: Actual binding is of type > > [ch.qos.logback.classic.util.ContextSelectorStaticBinder] > > [WARNING] > > java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native = Method) > > at > > = sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 62) > > at > > = sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:497) > > at > > = org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRu= nMojo.java:467) > > at java.lang.Thread.run(Thread.java:745) > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/flink/runtime/state/CheckpointListener > > at java.lang.ClassLoader.defineClass1(Native Method) > > at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > > at > > = java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > > at = java.net.URLClassLoader.defineClass(URLClassLoader.java:467) > > at = java.net.URLClassLoader.access$100(URLClassLoader.java:73) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:368) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > > at java.security.AccessController.doPrivileged(Native = Method) > > at = java.net.URLClassLoader.findClass(URLClassLoader.java:361) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > at java.lang.ClassLoader.defineClass1(Native Method) > > at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > > at > > = java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > > at = java.net.URLClassLoader.defineClass(URLClassLoader.java:467) > > at = java.net.URLClassLoader.access$100(URLClassLoader.java:73) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:368) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > > at java.security.AccessController.doPrivileged(Native = Method) > > at = java.net.URLClassLoader.findClass(URLClassLoader.java:361) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > at = com.dataartisans.ReadFromKafka.main(ReadFromKafka.java:53) > > ... 6 more > > Caused by: java.lang.ClassNotFoundException: > > org.apache.flink.runtime.state.CheckpointListener > > at = java.net.URLClassLoader.findClass(URLClassLoader.java:381) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > ... 29 more > > ``` > > > > My guess is that this occurs in the try block of the constructor > > `FlinkKafkaConsumer09(List topics, = KeyedDeserializationSchema > > deserializer, Properties props)`, and is probably due to me missing > > something in the classpath. Any hints on how to proceed would be > > appreciated. > > > > > > Prez Cannady > > p: 617 500 3378 > > e: revprez@opencorrelate.org > > GH: https://github.com/opencorrelate = > > LI: https://www.linkedin.com/in/revprez = > > > > > > > > > > > > > > > > > > >=20 --Apple-Mail=_1D1B4C03-213B-4B44-BEDE-5C4DB7C7C7E8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 I=E2=80=99ll give that a shot, but I should also report that = as of yesterday I=E2=80=99ve been able to get it to work with this = configuration:

https://github.com/OCExercise/kafka-example/tree/example-2.11-0= .9.0.1

My = primary blocking issue seems to have been attempting to publish and = subscribe to an older (~0.8.x.x) Kafka instance.






On Mar 9, 2016, at 6:31 AM, Stephan Ewen <sewen@apache.org> = wrote:

Hi!

Can you use version 1.0.0 for everything? That should make = all dependencies consistent.

Greetings,
Stephan


On Wed, = Mar 9, 2016 at 11:39 AM, Maximilian Michels <mxm@apache.org> wrote:
Hi Prez,

It appears Spring's Classloader is not set up correctly.
Unfortunately, I'm not familiar with the way Springboot works.

You added flink-connector-kafka-0.9_2.10 but also mentioned you're
using Scala 2.11. That is bound to cause troubles :)

Cheers,
Max

On Thu, Mar 3, 2016 at 8:02 PM, Prez Cannady
<revprez@correlatesystems.com> wrote:
> I=E2=80=99ve forked and am now experimenting with Robert = Metzler=E2=80=99s kaka-example.
>
> https://github.com/OCExercise/kafka-example
>
> Work fine from the vanilla fork (on the master branch).  I = performed my
> changes on branch enerscore-2.11, which includes:
>
> 1. Going from Flink 0.9 to Flink 0.10.2.
> 2. Using flink-connector-kafka-0.9_2.10 1.1-SNAPSHOT (which brings = in Kafka
> 0.9).
> 3. Removed the shade plugin.
> 4. Using Scala 2.11
> 5. Instrumenting with spring-boot.
>
> Currently running into this problem while attempting to invoke `mvn = clean
> spring-boot:run`:
>
> ```
> [INFO] --- spring-boot-maven-plugin:1.3.1.RELEASE:run (default-cli) = @
> kafka-example ---
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> = [jar:file:/Users/revprez/.m2/repository/ch/qos/logback/logback-classic/1.1= .3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> = [jar:file:/Users/revprez/.m2/repository/org/slf4j/slf4j-log4j12/1.7.13/slf= 4j-log4j12-1.7.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for = an
> explanation.
> SLF4J: Actual binding is of type
> [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
> [WARNING]
> java.lang.reflect.InvocationTargetException
>         at = sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
= >         at
> = sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 62)
>         at
> = sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:43)
>         at = java.lang.reflect.Method.invoke(Method.java:497)
>         at
> = org.springframework.boot.maven.AbstractRunMojo$LaunchRunner.run(AbstractRu= nMojo.java:467)
>         at = java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/flink/runtime/state/CheckpointListener
>         at = java.lang.ClassLoader.defineClass1(Native Method)
>         at = java.lang.ClassLoader.defineClass(ClassLoader.java:760)
>         at
> = java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >         at = java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>         at = java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>         at = java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>         at = java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>         at = java.security.AccessController.doPrivileged(Native Method)
>         at = java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>         at = java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at = java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at = java.lang.ClassLoader.defineClass1(Native Method)
>         at = java.lang.ClassLoader.defineClass(ClassLoader.java:760)
>         at
> = java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >         at = java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>         at = java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>         at = java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>         at = java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>         at = java.security.AccessController.doPrivileged(Native Method)
>         at = java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>         at = java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at = java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at = com.dataartisans.ReadFromKafka.main(ReadFromKafka.java:53)
>         ... 6 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.runtime.state.CheckpointListener
>         at = java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>         at = java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at = java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         ... 29 more
> ```
>
> My guess is that this occurs in the try block of the constructor
> `FlinkKafkaConsumer09(List<String> topics, = KeyedDeserializationSchema<T>
> deserializer, Properties props)`, and is probably due to me = missing
> something in the classpath.  Any hints on how to proceed would = be
> appreciated.
>
>
> Prez Cannady
> p: 617 500 3378
> e: revprez@opencorrelate.org
> GH: https://github.com/opencorrelate
> LI: https://www.linkedin.com/in/revprez
>
>
>
>
>
>
>
>
>


= --Apple-Mail=_1D1B4C03-213B-4B44-BEDE-5C4DB7C7C7E8--