flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: NoClassDefFoundError with ElasticsearchSink on Yarn
Date Thu, 01 Sep 2016 09:01:18 GMT
Hi Steffen,

this looks like a Guava version mismatch to me.
Are you running exactly the same program on your local machine or did you
add dependencies to run it on the cluster (e.g. Kinesis).
Maybe Kinesis and Elasticsearch are using different Guava versions?

Best, Fabian

2016-09-01 10:45 GMT+02:00 Hausmann, Steffen <shausma@amazon.de>:

> Hi there,
>
> I’m running a flink program that reads from a Kinesis stream and
> eventually writes to an Elasticsearch2 sink. When I’m running the program
> locally from the IDE, everything seems to work fine, but when I’m executing
> the same program on an EMR cluster with Yarn, a NoClassDefFoundError occurs:
>
> java.lang.NoSuchMethodError: com.google.common.util.concurr
> ent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor;
>
>        at org.elasticsearch.threadpool.ThreadPool.<clinit>(ThreadPool.
> java:190)
>
>        at org.elasticsearch.client.transport.TransportClient$Builder.
> build(TransportClient.java:133)
>
>        at org.apache.flink.streaming.connectors.elasticsearch2.Elastic
> searchSink.open(ElasticsearchSink.java:164)
>
>        at org.apache.flink.api.common.functions.util.FunctionUtils.ope
> nFunction(FunctionUtils.java:38)
>
>        at org.apache.flink.streaming.api.operators.AbstractUdfStreamOp
> erator.open(AbstractUdfStreamOperator.java:91)
>
>        at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllO
> perators(StreamTask.java:376)
>
>        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(
> StreamTask.java:256)
>
>        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
>
>        at java.lang.Thread.run(Thread.java:745)
>
> I’ve installed flink on an EMR cluster from the binary distribution
> flink-1.1.1-bin-hadoop27-scala_2.10.tgz. I build the jar file locally
> with mvn clean package(I’ve attached the pom.xml for reference) and copy it
> to the cluster for execution. There is a thread on this list that seems to
> be related, but I’m afraid I couldn’t draw any conclusions from it:
>
> http://apache-flink-user-mailing-list-archive.2336050.n4.
> nabble.com/classpath-issue-on-yarn-tt6442.html#none
>
>
> Any idea, what’s wrong?
>
> Thanks,
>
> Steffen
>
>

Mime
View raw message