flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Nowojski <pi...@data-artisans.com>
Subject Re: Akka version conflict running on Flink cluster
Date Mon, 11 Jun 2018 11:26:30 GMT
Hi,

Please take a look on this thread first:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Akka-Http-used-in-custom-RichSourceFunction-td20314.html
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Akka-Http-used-in-custom-RichSourceFunction-td20314.html>

Piotrek

> On 11 Jun 2018, at 11:16, Wouter Zorgdrager <zorgdragerw@gmail.com> wrote:
> 
> Hi,
> 
> I think I'm running into an Akka version conflict when running a Flink job on a cluster.
> 
> The current situation:
> - Flink cluster on Flink 1.4.2 (using Docker)
> - Flink job which uses twitter4s [1] library and Akka version 2.5.8
> 
> In my Flink job I try to 'shutdown' an Akka actor from the twitter4s library.
> This results in a whole taskmanager crashing with the following stacktrace:
> 
> taskrunner_1  | 2018-06-11 09:03:14,454 INFO  org.apache.flink.runtime.taskmanager.TaskManager
             - Un-registering task and sending final execution state CANCELED to JobManager
for task Source: Custom Source -> Sink: Unnamed (0ba7f7f259eee06fe2f7d783c868179b)
> taskrunner_1  | Uncaught error from thread [twitter4s-streaming-akka.actor.default-dispatcher-288]:
loader constraint violation: when resolving method "akka.actor.ActorCell$$anonfun$3.<init>(Lakka/actor/ActorCell;)V"
the class loader (instance of org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
of the current class, akka/actor/ActorCell, and the class loader (instance of sun/misc/Launcher$AppClassLoader)
for the method's defining class, akka/actor/ActorCell$$anonfun$3, have different Class objects
for the type akka/actor/ActorCell used in the signature, shutting down JVM since 'akka.jvm-exit-on-fatal-error'
is enabled for for ActorSystem[twitter4s-streaming]
> taskrunner_1  | java.lang.LinkageError: loader constraint violation: when resolving method
"akka.actor.ActorCell$$anonfun$3.<init>(Lakka/actor/ActorCell;)V" the class loader (instance
of org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
of the current class, akka/actor/ActorCell, and the class loader (instance of sun/misc/Launcher$AppClassLoader)
for the method's defining class, akka/actor/ActorCell$$anonfun$3, have different Class objects
for the type akka/actor/ActorCell used in the signature
> taskrunner_1  |     at akka.actor.ActorCell.invoke(ActorCell.scala:499)
> taskrunner_1  |     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> taskrunner_1  |     at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> taskrunner_1  |     at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> taskrunner_1  |     at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> taskrunner_1  |     at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> taskrunner_1  |     at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> taskrunner_1  |     at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> taskrunner_1  | 2018-06-11 09:03:14,984 INFO  org.apache.flink.runtime.blob.PermanentBlobCache
             - Shutting down BLOB cache
> taskrunner_1  | 2018-06-11 09:03:14,985 INFO  org.apache.flink.runtime.blob.TransientBlobCache
             - Shutting down BLOB cache
> taskrunner_1  | Exception in thread "twitter4s-streaming-shutdown-hook-1" java.lang.NoClassDefFoundError:
akka/actor/CoordinatedShutdown$$anonfun$totalTimeout$1
> taskrunner_1  |     at akka.actor.CoordinatedShutdown.totalTimeout(CoordinatedShutdown.scala:515)
> taskrunner_1  |     at akka.actor.CoordinatedShutdown$$anonfun$initJvmHook$1.apply(CoordinatedShutdown.scala:217)
> taskrunner_1  |     at akka.actor.CoordinatedShutdown$$anon$2.run(CoordinatedShutdown.scala:547)
> taskrunner_1  | Caused by: java.lang.ClassNotFoundException: akka.actor.CoordinatedShutdown$$anonfun$totalTimeout$1
> taskrunner_1  |     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> taskrunner_1  |     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> taskrunner_1  |     at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:128)
> taskrunner_1  |     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> taskrunner_1  |     ... 3 more
> 
> To me, it looks like an version conflict. Any suggestions how to solve this?
> 
> Thanks!
> Wouter
> 
> [1] - Twitter4s: https://github.com/DanielaSfregola/twitter4s/blob/master/build.sbt <https://github.com/DanielaSfregola/twitter4s/blob/master/build.sbt>
> 


Mime
View raw message