kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 凡梦星尘 <elkan1...@gmail.com>
Subject Kylin2.0.0在Spark模式下报Class未注册的问题
Date Tue, 06 Jun 2017 02:00:09 GMT
Hi guys:

按照官方的参考:https://kylin.apache.org/docs20/tutorial/cube_spark.html
 尝试最新的版本的KYLIN2.0.0版本下的Spark模式,配置启动都是OK,导入sample数据也可以正常构建CUBE,但是当构建自有数据时就失败了,异常的堆栈信息如下,这是个BUG吗?目前有没解决的办法呢?

Job aborted due to stage failure: Task 1 in stage 4.0 failed 4 times,
most recent failure: Lost task 1.3 in stage 4.0 (TID 29,
test-hdp-cluster01): com.esotericsoftware.kryo.KryoException:
java.lang.IllegalArgumentException: Class is not registered:
org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MappeableBitmapContainer
Note: To register this class use:
kryo.register(org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MappeableBitmapContainer.class);
Serialization trace:
values (org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MutableRoaringArray)
highLowContainer
(org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MutableRoaringBitmap)
bitmap (org.apache.kylin.measure.bitmap.RoaringBitmapCounter)
	at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
	at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
	at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
	at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
	at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
	at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)
	at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:318)
	at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:293)
	at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)
	at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:37)
	at com.twitter.chill.Tuple2Serializer.write(TupleSerializers.scala:33)
	at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)
	at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194)
	at org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:153)
	at org.apache.spark.storage.BlockManager.dataSerializeStream(BlockManager.scala:1252)
	at org.apache.spark.storage.BlockManager.dataSerialize(BlockManager.scala:1258)
	at org.apache.spark.storage.MemoryStore.putArray(MemoryStore.scala:136)
	at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:811)
	at org.apache.spark.storage.BlockManager.putArray(BlockManager.scala:687)
	at org.apache.spark.CacheManager.putInBlockManager(CacheManager.scala:175)
	at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:78)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:89)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Class is not
registered: org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MappeableBitmapContainer
Note: To register this class use:
kryo.register(org.apache.kylin.job.shaded.org.roaringbitmap.buffer.MappeableBitmapContainer.class);
	at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:442)
	at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:79)
	at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:472)
	at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:565)
	at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:318)
	at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:293)
	at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:501)
	at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564)
	... 33 more

Driver stacktrace:

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message