spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Rudenko <petro.rude...@gmail.com>
Subject [Tungsten] NPE in UnsafeShuffleWriter.java
Date Fri, 19 Jun 2015 18:36:07 GMT
Hi want to try new tungsten-sort shuffle manager, but on 1 stage 
executors start to die with NPE:

15/06/19 17:53:35 WARN TaskSetManager: Lost task 38.0 in stage 41.0 (TID 
3176, ip-10-50-225-214.ec2.internal): java.lang.NullPointerException
         at 
org.apache.spark.shuffle.unsafe.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:151)
         at 
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)
         at 
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
         at org.apache.spark.scheduler.Task.run(Task.scala:70)
         at 
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
         at java.lang.Thread.run(Thread.java:745)


Any suggestions?

Thanks,
Peter Rudenko

Mime
View raw message