flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bart van Deenen <bartvandee...@fastmail.fm>
Subject jackson DefaultScalaModule missing on deployment
Date Tue, 29 Mar 2016 11:17:42 GMT
Hi all

I've succesfully built a Flink streaming job, and it runs beautifully in
my IntelliJ ide, with a Flink instance started on the fly. The job eats
Kafka events, and outputs to file. All the i/o is json encoded with
Jackson.

But I'm having trouble with deploying the jar on a Flink server 
(version 1.0.0)
Building the jar via sbt package generates a jar with only my classes in
it, and the Flink cluster misses some Jackson stuff, and it complains
when I try to run the jar file.

java.lang.NoClassDefFoundError:
com/fasterxml/jackson/module/scala/DefaultScalaModule

The build.sbt is pretty much identical to the quick-start:

resolvers in ThisBuild ++= Seq(Resolver.mavenLocal)
name := "Flink Project"
version := "0.1-SNAPSHOT"
organization := "org.example"
scalaVersion in ThisBuild := "2.11.7"
val flinkVersion = "1.0.0"

val flinkDependencies = Seq(
  "org.apache.flink" %% "flink-scala" % flinkVersion % "provided",
  "org.apache.flink" %% "flink-streaming-scala" % flinkVersion %
  "provided")
lazy val root = (project in file(".")).
  settings(
    libraryDependencies ++= flinkDependencies
  )
libraryDependencies += "org.apache.flink" %
"flink-connector-kafka-0.9_2.11" % "1.0.0"
libraryDependencies += "com.fasterxml.jackson.module" %
"jackson-module-scala_2.11" % "2.4.5"
mainClass in assembly := Some("com.kpn.cj.journeyengine.Streaming")

// make run command include the provided dependencies
run in Compile <<= Defaults.runTask(fullClasspath in Compile, mainClass
in (Compile, run), runner in (Compile, run))

// exclude Scala library from assembly
assemblyOption in assembly := (assemblyOption in
assembly).value.copy(includeScala = false)

I'm not a hero as far as Scala Build Tool is concerned, any hints on how
to make sure the jackson stuff is actually included in the flink job?

Thanks

Bart

Mime
View raw message