spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "holdenk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-8498) Fix NullPointerException in error-handling path in UnsafeShuffleWriter
Date Sat, 20 Jun 2015 07:21:01 GMT

    [ https://issues.apache.org/jira/browse/SPARK-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594452#comment-14594452
] 

holdenk commented on SPARK-8498:
--------------------------------

I could take this :)

> Fix NullPointerException in error-handling path in UnsafeShuffleWriter
> ----------------------------------------------------------------------
>
>                 Key: SPARK-8498
>                 URL: https://issues.apache.org/jira/browse/SPARK-8498
>             Project: Spark
>          Issue Type: Bug
>          Components: Shuffle
>    Affects Versions: 1.4.0
>            Reporter: Josh Rosen
>
> This bug was reported by [~prudenko] on the dev list.  When the {{tungsten-sort}} shuffle
manager was enabled, an executor died with the following exception:
> {code}
> 15/06/19 17:53:35 WARN TaskSetManager: Lost task 38.0 in stage 41.0 (TID 3176, ip-10-50-225-214.ec2.internal):
java.lang.NullPointerException
>         at org.apache.spark.shuffle.unsafe.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:151)
>         at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)
>         at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
>         at org.apache.spark.scheduler.Task.run(Task.scala:70)
>         at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> {code}
> I think that this is actually due to an error-handling issue.  In the stack trace, the
NPE is being thrown from an error-handling branch of a `finally` block:
> {code}
> public void write(scala.collection.Iterator<Product2<K, V>> records) throws
IOException {
>     boolean success = false;
>     try {
>       while (records.hasNext()) {
>         insertRecordIntoSorter(records.next());
>       }
>       closeAndWriteOutput();
>       success = true;
>     } finally {
>       if (!success) {
>         sorter.cleanupAfterError();  // <---- this is the line throwing the error
>       }
>     }
>   }
> {code}
> I suspect that what's happening is that an exception is being thrown from user / upstream
code in the initial call to records.next(), but the error-handling block is failing because
sorter == null since we haven't initialized it yet.
> We should fix this bug with a {{sorter != null}} check and should also add a regression
test to ShuffleSuite to ensure that exceptions thrown by user code at this step of the shuffle
write path don't get masked by error-handling bugs inside of the shuffle code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message