spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [SPARK-10554] [CORE] Fix NPE with ShutdownHook
Date Sat, 12 Sep 2015 08:51:07 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.5 5bf403c53 -> f8909a6c6


[SPARK-10554] [CORE] Fix NPE with ShutdownHook

https://issues.apache.org/jira/browse/SPARK-10554

Fixes NPE when ShutdownHook tries to cleanup temporary folders

Author: Nithin Asokan <Nithin.Asokan@Cerner.com>

Closes #8720 from nasokan/SPARK-10554.

(cherry picked from commit 8285e3b0d3dc0eff669eba993742dfe0401116f9)
Signed-off-by: Sean Owen <sowen@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f8909a6c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f8909a6c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f8909a6c

Branch: refs/heads/branch-1.5
Commit: f8909a6c67420c27570b1691268a965c032ba32d
Parents: 5bf403c
Author: Nithin Asokan <Nithin.Asokan@Cerner.com>
Authored: Sat Sep 12 09:50:49 2015 +0100
Committer: Sean Owen <sowen@cloudera.com>
Committed: Sat Sep 12 09:51:02 2015 +0100

----------------------------------------------------------------------
 .../main/scala/org/apache/spark/storage/DiskBlockManager.scala   | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/f8909a6c/core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala b/core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala
index 3f8d26e..f7e84a2 100644
--- a/core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala
+++ b/core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala
@@ -164,7 +164,9 @@ private[spark] class DiskBlockManager(blockManager: BlockManager, conf:
SparkCon
 
   private def doStop(): Unit = {
     // Only perform cleanup if an external service is not serving our shuffle files.
-    if (!blockManager.externalShuffleServiceEnabled || blockManager.blockManagerId.isDriver)
{
+    // Also blockManagerId could be null if block manager is not initialized properly.
+    if (!blockManager.externalShuffleServiceEnabled ||
+      (blockManager.blockManagerId != null && blockManager.blockManagerId.isDriver))
{
       localDirs.foreach { localDir =>
         if (localDir.isDirectory() && localDir.exists()) {
           try {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message