Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 63EC91857A for ; Tue, 23 Jun 2015 13:32:53 +0000 (UTC) Received: (qmail 85704 invoked by uid 500); 23 Jun 2015 13:32:53 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 85628 invoked by uid 500); 23 Jun 2015 13:32:53 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 85618 invoked by uid 99); 23 Jun 2015 13:32:53 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Jun 2015 13:32:53 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 8ED911821DA for ; Tue, 23 Jun 2015 13:32:52 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.88 X-Spam-Level: *** X-Spam-Status: No, score=3.88 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_REPLY=1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id ON_rJYIqa7qM for ; Tue, 23 Jun 2015 13:32:46 +0000 (UTC) Received: from mail-wi0-f173.google.com (mail-wi0-f173.google.com [209.85.212.173]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 20F562315E for ; Tue, 23 Jun 2015 13:32:46 +0000 (UTC) Received: by wicnd19 with SMTP id nd19so106050491wic.1 for ; Tue, 23 Jun 2015 06:31:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=yDZeo5AmL0NDMFwKFzQh19NZpNbAQlYTLKTfQJqZ2wU=; b=OqEO+pnKcxS0apM6l1yPTVlIdqK4G5boXPE040bsl8RC6ePqhejGzENPpgQE6RYCK4 8juNAkBMprt2UYLOtviAEnLBXPD9fgjOfyzx64Rvz/DLX5ycPgejmmMRRXy020jGp+Ou Qb9Jmhs4IB/n3tb77uCcV6rqVLGwr2l77PiQ6ab1YEjcxWaaMUjSn6XSLfD7RHdWP3il OJX25k5vDFYUVjNl+f9lOxTAhXntx2UTjdfXNANOGLFcoRs+yfT5lTkH1aLjMuBZfIU8 CorjgFKz6ys312YUfn/3vWpWHdX1gsNGshUBQ/Nau0UfOAyfNMFSOcIH552hUyT4BkH4 E/ww== X-Received: by 10.180.211.196 with SMTP id ne4mr3579185wic.23.1435066315035; Tue, 23 Jun 2015 06:31:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.86.165 with HTTP; Tue, 23 Jun 2015 06:31:35 -0700 (PDT) In-Reply-To: References: <55793c26.a936460a.3c9e.ffffc1b1@mx.google.com> <55794102.8340460a.6d2c.ffffca2a@mx.google.com> From: =?UTF-8?Q?M=C3=A1rton_Balassi?= Date: Tue, 23 Jun 2015 15:31:35 +0200 Message-ID: Subject: Re: Kafka0.8.2.1 + Flink0.9.0 issue To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a11c33af4efdae705192f6953 --001a11c33af4efdae705192f6953 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Dear Hawin, Sorry, I ahve managed to link to a pom that has been changed in the meantime. But we have added a section to our doc clarifying your question. [1] Since then Stephan has proposed an even nicer solution that did not make it into the doc yet, namely if you start from our quickstart pom and add your dependencies to that simply executing 'mvn package -Pbuild-jar' you get a jar with all your the code that is needed to run it on the cluster, but not more. See [3] for more on the quickstart. [1] http://ci.apache.org/projects/flink/flink-docs-master/apis/cluster_executio= n.html#linking-with-modules-not-contained-in-the-binary-distribution [2] https://github.com/apache/flink/blob/master/flink-quickstart/flink-quicksta= rt-java/src/main/resources/archetype-resources/pom.xml [3] http://ci.apache.org/projects/flink/flink-docs-master/quickstart/java_api_q= uickstart.html On Tue, Jun 23, 2015 at 6:48 AM, Ashutosh Kumar wrote: > I use following dependencies and it works fine . > > > org.apache.flink > flink-java > 0.9-SNAPSHOT > > > org.apache.flink > flink-clients > 0.9-SNAPSHOT > > > org.apache.flink > flink-streaming-core > 0.9-SNAPSHOT > > > org.apache.flink > flink-connector-kafka > 0.9-SNAPSHOT > > > > On Mon, Jun 22, 2015 at 10:07 PM, Hawin Jiang > wrote: > >> Hi Marton >> >> I have to add whole pom.xml file or just only plugin as below. >> I saw L286 to L296 are not correct information in pom.xml. >> Thanks. >> >> >> >> org.apache.maven.plugins > >maven-assembly-plugin 2.4 > > jar-with-dependencies >> >> >> On Thu, Jun 11, 2015 at 1:43 AM, M=C3=A1rton Balassi > > wrote: >> >>> As for locally I meant the machine that you use for development to see >>> whether this works without parallelism. :-) No need to install stuff on >>> your Namenode of course. >>> Installing Kafka on a machine and having the Kafka Java dependencies >>> available for Flink are two very different things. Try adding the follo= wing >>> [1] to your maven pom. Then execute 'mvn assembly:assembly', this will >>> produce a fat jar suffiexed jar-with-dependencies.jar. You should be ab= le >>> to run the example form that. >>> >>> [1] >>> https://github.com/mbalassi/flink-dataflow/blob/master/pom.xml#L286-296 >>> >>> On Thu, Jun 11, 2015 at 10:32 AM, Hawin Jiang >>> wrote: >>> >>>> Dear Marton >>>> >>>> What do you meaning for locally Eclipse with 'Run'. >>>> Do you want to me to run it on Namenode? >>>> But my namenode didn't install Kafka. I only installed Kafka on my >>>> data node servers. >>>> Do I need to install or copy Kafka jar on Namenode? Actually, I don't >>>> want to install everything on Name node server. >>>> Please advise me. >>>> Thanks. >>>> >>>> >>>> My Flink and Hadoop cluster info as below. >>>> >>>> Flink on NameNode >>>> Kafka,Zookeeper and FLink slave1 on Datanode1 >>>> Kafka,Zookeeper ,and Flink slave2 on Datanode2 >>>> Kafka, Zookeeper and Flink slave3 on Datanode3 >>>> >>>> >>>> >>>> On Thu, Jun 11, 2015 at 1:16 AM, M=C3=A1rton Balassi < >>>> balassi.marton@gmail.com> wrote: >>>> >>>>> Dear Hawin, >>>>> >>>>> No problem, I am gald that you are giving our Kafka connector a try. = :) >>>>> The dependencies listed look good. Can you run the example locally >>>>> from Eclipse with 'Run'? I suspect that maybe your Flink cluster does= not >>>>> have the access to the kafka dependency then. >>>>> >>>>> As a quick test you could copy the kafka jars to the lib folder of >>>>> your Flink distribution on all the machines in your cluster. Everythi= ng >>>>> that is there goes to the classpath of Flink. Another workaround with= be to >>>>> build a fat jar for your project containing all the dependencies with= 'mvn >>>>> assembly:assembly'. Neither of these are beautiful but would help tra= cking >>>>> down the root cause. >>>>> >>>>> On Thu, Jun 11, 2015 at 10:04 AM, Hawin Jiang >>>>> wrote: >>>>> >>>>>> Dear Marton >>>>>> >>>>>> >>>>>> >>>>>> Thanks for supporting again. >>>>>> >>>>>> I am running these examples at the same project and I am using >>>>>> Eclipse IDE to submit it to my Flink cluster. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Here is my dependencies >>>>>> >>>>>> >>>>>> ********************************************************************= ********** >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *junit* >>>>>> >>>>>> *junit* >>>>>> >>>>>> 4.12 >>>>>> >>>>>> test >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.flink >>>>>> >>>>>> *flink*-java >>>>>> >>>>>> 0.9.0-milestone-1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.flink >>>>>> >>>>>> *flink*-clients >>>>>> >>>>>> 0.9.0-milestone-1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.flink >>>>>> >>>>>> *flink*-streaming-connectors >>>>>> >>>>>> 0.9.0-milestone-1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.flink >>>>>> >>>>>> *flink*-streaming-core >>>>>> >>>>>> 0.9.0-milestone-1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.kafka >>>>>> >>>>>> kafka_2.10 >>>>>> >>>>>> 0.8.2.1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.kafka >>>>>> >>>>>> *kafka*-clients >>>>>> >>>>>> 0.8.2.1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.hadoop >>>>>> >>>>>> *hadoop*-*hdfs* >>>>>> >>>>>> 2.6.0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.hadoop >>>>>> >>>>>> *hadoop*-*auth* >>>>>> >>>>>> 2.6.0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.hadoop >>>>>> >>>>>> *hadoop*-common >>>>>> >>>>>> 2.6.0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> org.apache.hadoop >>>>>> >>>>>> *hadoop*-core >>>>>> >>>>>> 1.2.1 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ********************************************************************= ********************* >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Best regards >>>>>> >>>>>> Email: hawin.jiang@gmail.com >>>>>> >>>>>> >>>>>> >>>>>> *From:* M=C3=A1rton Balassi [mailto:balassi.marton@gmail.com] >>>>>> *Sent:* Thursday, June 11, 2015 12:58 AM >>>>>> *To:* user@flink.apache.org >>>>>> *Subject:* Re: Kafka0.8.2.1 + Flink0.9.0 issue >>>>>> >>>>>> >>>>>> >>>>>> Dear Hawin, >>>>>> >>>>>> >>>>>> >>>>>> This looks like a dependency issue, the java compiler does not find >>>>>> the kafka dependency. How are you trying to run this example? Is it = from an >>>>>> IDE or submitting it to a flink cluster with bin/flink run? How do y= ou >>>>>> define your dependencies, do you use maven or sbt for instance? >>>>>> >>>>>> >>>>>> >>>>>> Best, >>>>>> >>>>>> >>>>>> >>>>>> Marton >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jun 11, 2015 at 9:43 AM, Hawin Jiang >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> > --001a11c33af4efdae705192f6953 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Dear Hawin,

Sorry, I ahve managed to li= nk to a pom that has been changed in the meantime. But we have added a sect= ion to our doc clarifying your question. [1] Since then Stephan has propose= d an even nicer solution that did not make it into the doc yet, namely if y= ou start from our quickstart pom and add your dependencies to that simply e= xecuting 'mvn package -Pbuild-jar' you get a jar with all your the = code that is needed to run it on the cluster, but not more. See [3] for mor= e on the quickstart.

[3]=C2=A0http://ci.apache.org/projects/flink/flink-docs-mas= ter/quickstart/java_api_quickstart.html

On Tue, Jun 23, 2015 at 6:48 AM, Ashu= tosh Kumar <ashutosh.discuss@gmail.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
I use following dependencies= and it works fine .

<dependenc= y>
<groupId&= gt;org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>0.9-S= NAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>fli= nk-clients</artifactId>
<version>0.9-SNAPSHOT</version>
</dependenc= y>
<dependenc= y>
<groupId&= gt;org.apache.flink</groupId>
<artifactId>flink-streaming-core</artifactId>=
<versio= n>0.9-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.flink</gro= upId>
&l= t;artifactId>flink-connector-kafka</artifactId>
<version>0.9-SNAPSHOT</vers= ion>
</depend= ency>
<depend= ency>

On Mon, Jun 22, 2015 at 10= :07 PM, Hawin Jiang <hawin.jiang@gmail.com> wrote:
Hi =C2=A0Marton

I have to add whole pom.xml file or just only plugin as below.=C2=A0<= /div>
I saw L286 to L296 are not correct information in pom.xml.
<= div>Thanks.



= <= /tr>=
<plugin>
<groupId>= org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifac= tId>
<version>2.4</version>
= <configuration>
= <descriptorRefs>
= <descriptorRef>jar-= with-dependencies</descriptorRef>
</descriptorRefs<= /span>>
</configuration<= /span>>
</plugin&g= t;

On Thu, Jun 11, 2015 at 1:43 AM, M=C3=A1rt= on Balassi <balassi.marton@gmail.com> wrote:
As for locally I meant the machi= ne that you use for development to see whether this works without paralleli= sm. :-) No need to install stuff on your Namenode of course.
Installing= Kafka on a machine and having the Kafka Java dependencies available for Fl= ink are two very different things. Try adding the following [1] to your mav= en pom. Then execute 'mvn assembly:assembly', this will produce a f= at jar suffiexed=C2=A0jar-with-dependencies.jar. You should be able to run = the example form that.


On Thu, Jun 11, 2015 at 10:32 AM, Hawin Jiang <hawin.jia= ng@gmail.com> wrote:
Dear Marton

What do you meaning for locally E= clipse with 'Run'. =C2=A0
Do you want to me to run it on = Namenode? =C2=A0
But my namenode didn't install Kafka.=C2=A0 = I only installed Kafka on my data node servers.=C2=A0
Do I need t= o install or copy Kafka jar on Namenode? Actually, I don't want to inst= all everything on Name node server.=C2=A0
Please advise me.
=
Thanks.=C2=A0


My Flink and Had= oop cluster info as below.

Flink on NameNode
=
Kafka,Zookeeper and FLink slave1 on Datanode1
Kafka,Zookeepe= r ,and Flink slave2 on Datanode2
Kafka,=C2=A0Zookeeper and Flink = slave3 on Datanode3



On Thu, Jun 11, 2015 = at 1:16 AM, M=C3=A1rton Balassi <balassi.marton@gmail.com> wrote:
Dear Hawin,=

No problem, I am gald that you are giving our Kafka con= nector a try. :)
The dependencies listed look good. Can you run t= he example locally from Eclipse with 'Run'? I suspect that maybe yo= ur Flink cluster does not have the access to the kafka dependency then.=C2= =A0

As a quick test you could copy the kafka jars = to the lib folder of your Flink distribution on all the machines in your cl= uster. Everything that is there goes to the classpath of Flink. Another wor= karound with be to build a fat jar for your project containing all the depe= ndencies with 'mvn assembly:assembly'. Neither of these are beautif= ul but would help tracking down the root cause.

On Thu, Jun 11, 2015 at= 10:04 AM, Hawin Jiang <hawin.jiang@gmail.com> wrote:

Dear Marton

=C2=A0

Thanks for supporting aga= in.

I am running these examples at the same project and I am = using Eclipse IDE to submit it to my Flink cluster.

= =C2=A0

=C2=A0

Here is my dependencies<= /span>

*******************************************= ***********************************

<dependencies<= /span>>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = <dependency>

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <= groupId>junit</= groupId>

=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <ar= tifactId>junit</artifactId>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <version>4.12</version>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 <scope>test</scope>=

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </d= ependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 <dependency>

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.flink</groupId><= /p>

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>flink-java</artifactId>

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:te= al"><version>0.9.0-milestone-= 1</version>

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </depend= ency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <= /span><dependency>

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 &l= t;groupId>org.apache.flink</groupId>

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>flink-clients</artifactId>=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <version>0.9.0-milesto= ne-1</version><= /span>

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </d= ependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 <dependency>

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.flink</groupId><= /p>

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>flink-streaming-connectors</artifactId<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:te= al">>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 <version>0.9.0-milestone-1</version><= /span>=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </dependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 <dependency><= /u>

<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:bl= ack">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.= flink</groupId>=

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>flink<= /u>-streaming-core</artifactId<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:te= al">>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 <version>0.9.0-milestone-1</version><= /span>=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </dependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 <dependency><= /u>

<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:bl= ack">=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.= kafka</groupId>=

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>kafka_2.10</artifactId><= /p>

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <version>0.8.2.1</version>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 </dependency>=

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <dependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.kafka</g= roupId>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 <artifactId<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:te= al">>kafka-clients</artifactId>

=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <ve= rsion>0.8.2.1</v= ersion>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = </dependency>

=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <depende= ncy>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.hadoop</groupId<= /span>>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 <artifactId>= ;hadoop-hdfs</artifactId>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <version= >2.6.0</version>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:te= al"></dependency>

=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 <dependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 <groupId><= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:bl= ack">org.apache.hadoop</groupId&g= t;

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 <artifactId>hadoop-auth</artifa= ctId>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 <version>2.6.0</version>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </= dependency>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 <dependency= >

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 <groupId>o= rg.apache.hadoop</groupId>=

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>hado= op-common</artifactId= >

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 <version><= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:bl= ack">2.6.0</version>

= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 </= dependency>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 <dependency>

=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <groupId>org.apache.hadoop</groupId>=

=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <artifactId>hadoop-core</artifactId><= u>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = <version>1.2.1</version>

=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 </dependency<= span lang=3D"EN-US" style=3D"font-size:10.0pt;font-family:Consolas;color:te= al">>

=C2=A0=C2=A0= =C2=A0 </dependencies><= u>

=C2=A0

**********************************************= *******************************************

=C2=A0

=C2=A0

<= u>=C2=A0

=C2=A0

Best regards

Email: hawin.jiang@gmail.com

=

=C2= =A0

From: M=C3=A1rton Balassi [ma= ilto:balassi.= marton@gmail.com]
Sent: Thursday, June 11, 2015 12:58 AM
= To: user@= flink.apache.org
Subject: Re: Kafka0.8.2.1 + Flink0.9.0 issue=

=C2=A0

= Dear Hawin,

=C2=A0

<= div>

This looks like a dependenc= y issue, the java compiler does not find the kafka dependency. How are you = trying to run this example? Is it from an IDE or submitting it to a flink c= luster with bin/flink run? How do you define your dependencies, do you use = maven or sbt for instance?

=C2=A0

Best,

=C2=A0

Marton

<= u>=C2=A0

On Thu, Jun 11, 2015 at 9:43 AM, Hawin Jiang <hawin.jiang@gmail.com> wrote:

=C2=A0







--001a11c33af4efdae705192f6953--