spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Lists <lists.a...@gmail.com>
Subject ClassNotFoundException when registering classes with Kryo
Date Mon, 02 Feb 2015 02:08:42 GMT
Here is the relevant snippet of code in my main program:

===========================================

sparkConf.set("spark.serializer",
  "org.apache.spark.serializer.KryoSerializer")
sparkConf.set("spark.kryo.registrationRequired", "true")
val summaryDataClass = classOf[SummaryData]
val summaryViewClass = classOf[SummaryView]
sparkConf.registerKryoClasses(Array(

  summaryDataClass, summaryViewClass))

===========================================

I get the following error:

Exception in thread "main" java.lang.reflect.InvocationTargetException
...

Caused by: org.apache.spark.SparkException: Failed to load class to
register with Kryo
...

Caused by: java.lang.ClassNotFoundException:
com.dtex.analysis.transform.SummaryData


Note that the class in question SummaryData is in the same package as the
main program and hence in the same jar.

What do I need to do to make this work?

Thanks,
arun

Mime
View raw message