flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Kafka0.8.2.1 + Flink0.9.0 issue
Date Thu, 11 Jun 2015 08:16:26 GMT
Dear Hawin,

No problem, I am gald that you are giving our Kafka connector a try. :)
The dependencies listed look good. Can you run the example locally from
Eclipse with 'Run'? I suspect that maybe your Flink cluster does not have
the access to the kafka dependency then.

As a quick test you could copy the kafka jars to the lib folder of your
Flink distribution on all the machines in your cluster. Everything that is
there goes to the classpath of Flink. Another workaround with be to build a
fat jar for your project containing all the dependencies with 'mvn
assembly:assembly'. Neither of these are beautiful but would help tracking
down the root cause.

On Thu, Jun 11, 2015 at 10:04 AM, Hawin Jiang <hawin.jiang@gmail.com> wrote:

> Dear Marton
>
>
>
> Thanks for supporting again.
>
> I am running these examples at the same project and I am using Eclipse IDE
> to submit it to my Flink cluster.
>
>
>
>
>
> Here is my dependencies
>
>
> ******************************************************************************
>
> <dependencies>
>
>         <dependency>
>
>             <groupId>*junit*</groupId>
>
>             <artifactId>*junit*</artifactId>
>
>             <version>4.12</version>
>
>             <scope>test</scope>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.flink</groupId>
>
>             <artifactId>*flink*-java</artifactId>
>
>             <version>0.9.0-milestone-1</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.flink</groupId>
>
>             <artifactId>*flink*-clients</artifactId>
>
>             <version>0.9.0-milestone-1</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.flink</groupId>
>
>             <artifactId>*flink*-streaming-connectors</artifactId>
>
>             <version>0.9.0-milestone-1</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.flink</groupId>
>
>             <artifactId>*flink*-streaming-core</artifactId>
>
>             <version>0.9.0-milestone-1</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.kafka</groupId>
>
>             <artifactId>kafka_2.10</artifactId>
>
>             <version>0.8.2.1</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.kafka</groupId>
>
>             <artifactId>*kafka*-clients</artifactId>
>
>             <version>0.8.2.1</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.hadoop</groupId>
>
>             <artifactId>*hadoop*-*hdfs*</artifactId>
>
>             <version>2.6.0</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.hadoop</groupId>
>
>             <artifactId>*hadoop*-*auth*</artifactId>
>
>             <version>2.6.0</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.hadoop</groupId>
>
>             <artifactId>*hadoop*-common</artifactId>
>
>             <version>2.6.0</version>
>
>         </dependency>
>
>         <dependency>
>
>             <groupId>org.apache.hadoop</groupId>
>
>             <artifactId>*hadoop*-core</artifactId>
>
>             <version>1.2.1</version>
>
>         </dependency>
>
>     </dependencies>
>
>
>
>
> *****************************************************************************************
>
>
>
>
>
>
>
>
>
> Best regards
>
> Email: hawin.jiang@gmail.com
>
>
>
> *From:* Márton Balassi [mailto:balassi.marton@gmail.com]
> *Sent:* Thursday, June 11, 2015 12:58 AM
> *To:* user@flink.apache.org
> *Subject:* Re: Kafka0.8.2.1 + Flink0.9.0 issue
>
>
>
> Dear Hawin,
>
>
>
> This looks like a dependency issue, the java compiler does not find the
> kafka dependency. How are you trying to run this example? Is it from an IDE
> or submitting it to a flink cluster with bin/flink run? How do you define
> your dependencies, do you use maven or sbt for instance?
>
>
>
> Best,
>
>
>
> Marton
>
>
>
> On Thu, Jun 11, 2015 at 9:43 AM, Hawin Jiang <hawin.jiang@gmail.com>
> wrote:
>
>
>

Mime
View raw message