spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From har...@nexgate.com
Subject Re: Spark Streaming from Kafka
Date Wed, 29 Oct 2014 07:31:07 GMT
Thanks! How do I find out which Kafka jar to use for scala 2.10.4?

—
Sent from Mailbox

On Wed, Oct 29, 2014 at 12:26 AM, Akhil Das <akhil@sigmoidanalytics.com>
wrote:

> Looks like the kafka jar that you are using isn't compatible with your
> scala version.
> Thanks
> Best Regards
> On Wed, Oct 29, 2014 at 11:50 AM, Harold Nguyen <harold@nexgate.com> wrote:
>> Hi,
>>
>> Just wondering if you've seen the following error when reading from Kafka:
>>
>> ERROR ReceiverTracker: Deregistered receiver for stream 0: Error starting
>> receiver 0 - java.lang.NoClassDefFoundError: scala/reflect/ClassManifest
>> at kafka.utils.Log4jController$.<init>(Log4jController.scala:29)
>> at kafka.utils.Log4jController$.<clinit>(Log4jController.scala)
>> at kafka.utils.Logging$class.$init$(Logging.scala:29)
>> at kafka.utils.VerifiableProperties.<init>(VerifiableProperties.scala:24)
>> at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:78)
>> at
>> org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:96)
>> at
>> org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:121)
>> at
>> org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:106)
>> at
>> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$9.apply(ReceiverTracker.scala:264)
>> at
>> org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverLauncher$$anonfun$9.apply(ReceiverTracker.scala:257)
>> at
>> org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
>> at
>> org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
>> at org.apache.spark.scheduler.Task.run(Task.scala:54)
>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:744)
>> Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassManifest
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> ... 18 more
>>
>> Thanks,
>>
>> Harold
>>
Mime
View raw message