spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-15604) Spark-SQL: Get com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException when runing query_1.sql of TPC-DS
Date Fri, 27 May 2016 21:10:14 GMT

     [ https://issues.apache.org/jira/browse/SPARK-15604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin updated SPARK-15604:
--------------------------------
    Affects Version/s:     (was: 2.1.0)

> Spark-SQL: Get com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException when runing query_1.sql of TPC-DS
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-15604
>                 URL: https://issues.apache.org/jira/browse/SPARK-15604
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>         Environment: sparkļ¼šMaster development branch
> hadoop:2.7.1
> hive:1.2.1
> spark:Standalone, 1 Master,3 Worker
>            Reporter: wenjing wang
>              Labels: performance
>
> When I run query_1.sql of TPC-DS like:
> bin/spark-sql --master spark://192.168.30.78:7077 -i /data/disk1/data/100G/query_1.sql
> The process fails with a exception throwing:
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> The error details are:
> {code}
> bin/spark-sql --master spark://192.168.30.78:7077 -i /data/disk1/data/100G/query_1.sql
> 16/05/27 06:16:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where 
> applicable
> 16/05/27 06:16:57 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so 
> recording the schema version 1.2.0
> 16/05/27 06:16:57 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
> 16/05/27 06:16:58 WARN AbstractHandler: No Server set for org.spark_project.jetty.server.handler.ErrorHandler@55a354d8
> 16/05/27 06:16:59 WARN SparkContext: Use an existing SparkContext, some configuration may not take effect.
> [Stage 9:=================================>                    (123 + 42) / 200]16/05/27 06:17:38 ERROR TaskResultGetter: Exception while 
> getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR SparkSQLDriver: Failed in [with customer_total_return as
> (select sr_customer_sk as ctr_customer_sk
> ,sr_store_sk as ctr_store_sk
> ,sum(SR_FEE) as ctr_total_return
> from store_returns
> ,date_dim
> where sr_returned_date_sk = d_date_sk
> and d_year =2000
> group by sr_customer_sk
> ,sr_store_sk)
>  select  c_customer_id
> from customer_total_return ctr1
> ,store
> ,customer
> where ctr1.ctr_total_return > (select avg(ctr_total_return)*1.2
> from customer_total_return ctr2
> where ctr1.ctr_store_sk = ctr2.ctr_store_sk)
> and s_store_sk = ctr1.ctr_store_sk
> and s_state = 'SD'
> and ctr1.ctr_customer_sk = c_customer_sk
> order by c_customer_id
> limit 100]
> org.apache.spark.SparkException: Job aborted due to stage failure: Exception while getting task result: 
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages
> (DAGScheduler.scala:1435)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
>         at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>         at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
>         at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
>         at scala.Option.foreach(Option.scala:257)
>         at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:806)
>         at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1644)
>         at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1603)
>         at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1592)
>         at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
>         at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
>         at org.apache.spark.SparkContext.runJob(SparkContext.scala:1872)
>         at org.apache.spark.SparkContext.runJob(SparkContext.scala:1935)
>         at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:974)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
>         at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
>         at org.apache.spark.rdd.RDD.reduce(RDD.scala:956)
>         at org.apache.spark.rdd.RDD$$anonfun$takeOrdered$1.apply(RDD.scala:1371)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
>         at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
>         at org.apache.spark.rdd.RDD.takeOrdered(RDD.scala:1358)
>         at org.apache.spark.sql.execution.TakeOrderedAndProjectExec.executeCollect(limit.scala:127)
>         at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:310)
>         at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:126)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:323)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
>         at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:409)
>         at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:425)
>         at org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:436)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:157)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
>         at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
>         at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
>         at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
>         at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> org.apache.spark.SparkException: Job aborted due to stage failure: Exception while getting task result: 
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages
> (DAGScheduler.scala:1435)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
>         at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
>         at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
>         at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
>         at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:806)
>         at scala.Option.foreach(Option.scala:257)
>         at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:806)
>         at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1644)
>         at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1603)
>         at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1592)
>         at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
>         at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
>         at org.apache.spark.SparkContext.runJob(SparkContext.scala:1872)
>         at org.apache.spark.SparkContext.runJob(SparkContext.scala:1935)
>         at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:974)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
>         at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
>         at org.apache.spark.rdd.RDD.reduce(RDD.scala:956)
>         at org.apache.spark.rdd.RDD$$anonfun$takeOrdered$1.apply(RDD.scala:1371)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
>         at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
>         at org.apache.spark.rdd.RDD.withScope(RDD.scala:357)
>         at org.apache.spark.rdd.RDD.takeOrdered(RDD.scala:1358)
>         at org.apache.spark.sql.execution.TakeOrderedAndProjectExec.executeCollect(limit.scala:127)
>         at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:310)
>         at org.apache.spark.sql.execution.QueryExecution.hiveResultString(QueryExecution.scala:126)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:63)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:323)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
>         at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:409)
>         at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:425)
>         at org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:436)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:157)
>         at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
>         at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
>         at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
>         at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
>         at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 38.0 in stage 10.0 (TID 900, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 40.0 in stage 10.0 (TID 902, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 5.0 in stage 10.0 (TID 867, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 32.0 in stage 10.0 (TID 894, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 26.0 in stage 10.0 (TID 888, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 11.0 in stage 10.0 (TID 873, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 8.0 in stage 10.0 (TID 870, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 2.0 in stage 10.0 (TID 864, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 ERROR TaskResultGetter: Exception while getting task result
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> underlying (org.apache.spark.util.BoundedPriorityQueue)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:25)
>         at com.twitter.chill.SomeSerializer.read(SomeSerializer.scala:19)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
>         at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:312)
>         at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:87)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:66)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2$$anonfun$run$1.apply(TaskResultGetter.scala:57)
>         at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1793)
>         at org.apache.spark.scheduler.TaskResultGetter$$anon$2.run(TaskResultGetter.scala:56)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:159)
>         at org.apache.spark.sql.catalyst.expressions.codegen.LazilyGeneratedOrdering.compare(GenerateOrdering.scala:150)
>         at scala.math.Ordering$$anon$4.compare(Ordering.scala:111)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:627)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:329)
>         at java.util.PriorityQueue.add(PriorityQueue.java:306)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:78)
>         at com.twitter.chill.java.PriorityQueueSerializer.read(PriorityQueueSerializer.java:31)
>         at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
>         at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
>         ... 15 more
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 29.0 in stage 10.0 (TID 891, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 37.0 in stage 10.0 (TID 899, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 42.0 in stage 10.0 (TID 904, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 17.0 in stage 10.0 (TID 879, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 22.0 in stage 10.0 (TID 885, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 20.0 in stage 10.0 (TID 882, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 45.0 in stage 10.0 (TID 907, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 35.0 in stage 10.0 (TID 897, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 39.0 in stage 10.0 (TID 901, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 43.0 in stage 10.0 (TID 905, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 14.0 in stage 10.0 (TID 876, 192.168.30.79): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 41.0 in stage 10.0 (TID 903, 192.168.30.72): TaskKilled (killed intentionally)
> 16/05/27 06:17:38 WARN TaskSetManager: Lost task 44.0 in stage 10.0 (TID 906, 192.168.30.72): TaskKilled (killed intentionally)
> {code}
> {code}
> query_1.sql
> with customer_total_return as
> (select sr_customer_sk as ctr_customer_sk
> ,sr_store_sk as ctr_store_sk
> ,sum(SR_FEE) as ctr_total_return
> from store_returns
> ,date_dim
> where sr_returned_date_sk = d_date_sk
> and d_year =2000
> group by sr_customer_sk
> ,sr_store_sk)
>  select  c_customer_id
> from customer_total_return ctr1
> ,store
> ,customer
> where ctr1.ctr_total_return > (select avg(ctr_total_return)*1.2
> from customer_total_return ctr2
> where ctr1.ctr_store_sk = ctr2.ctr_store_sk)
> and s_store_sk = ctr1.ctr_store_sk
> and s_state = 'SD'
> and ctr1.ctr_customer_sk = c_customer_sk
> order by c_customer_id
> limit 100;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message