flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steffen Hausmann <stef...@hausmann-family.de>
Subject NoClassDefFoundError with ElasticsearchSink on Yarn
Date Thu, 01 Sep 2016 08:58:49 GMT
Hi there,

I’m running a flink program that reads from a Kinesis stream and 
eventually writes to an Elasticsearch2 sink. When I’m running the 
program locally from the IDE, everything seems to work fine, but when 
I’m executing the same program on an EMR cluster with Yarn, a 
NoClassDefFoundError occurs:java.lang.NoSuchMethodError:

com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor;
        at 
org.elasticsearch.threadpool.ThreadPool.<clinit>(ThreadPool.java:190)
        at 
org.elasticsearch.client.transport.TransportClient$Builder.build(TransportClient.java:133)
        at 
org.apache.flink.streaming.connectors.elasticsearch2.ElasticsearchSink.open(ElasticsearchSink.java:164)
        at 
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:38)
        at 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:91)
        at 
org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:376)
        at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:256)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
        at java.lang.Thread.run(Thread.java:745)

I’ve installed flink on an EMR cluster from the binary distribution 
flink-1.1.1-bin-hadoop27-scala_2.10.tgzand the jar file that is executed 
on the cluster is build with mvn clean package(I’ve attached the pom.xml 
for reference).

There is a thread on this list that seems to be related, but I’m afraid 
I couldn’t draw any conclusions from it: 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/classpath-issue-on-yarn-tt6442.html#none

Any idea, what’s wrong?

Thanks,
Steffen


Mime
View raw message