spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wenc...@apache.org
Subject spark git commit: [SPARK-21953] Show both memory and disk bytes spilled if either is present
Date Mon, 18 Sep 2017 02:43:36 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-2.1 3ae7ab8e8 -> 99de4b8f5


[SPARK-21953] Show both memory and disk bytes spilled if either is present

As written now, there must be both memory and disk bytes spilled to show either of them. If
there is only one of those types of spill recorded, it will be hidden.

Author: Andrew Ash <andrew@andrewash.com>

Closes #19164 from ash211/patch-3.

(cherry picked from commit 6308c65f08b507408033da1f1658144ea8c1491f)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/99de4b8f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/99de4b8f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/99de4b8f

Branch: refs/heads/branch-2.1
Commit: 99de4b8f55ea2f700ad4ac32620217fa43a2cbdb
Parents: 3ae7ab8
Author: Andrew Ash <andrew@andrewash.com>
Authored: Mon Sep 18 10:42:24 2017 +0800
Committer: Wenchen Fan <wenchen@databricks.com>
Committed: Mon Sep 18 10:43:21 2017 +0800

----------------------------------------------------------------------
 core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/99de4b8f/core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala b/core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala
index 78113ac..cd7b396 100644
--- a/core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala
+++ b/core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala
@@ -104,7 +104,7 @@ private[spark] object UIData {
     def hasOutput: Boolean = outputBytes > 0
     def hasShuffleRead: Boolean = shuffleReadTotalBytes > 0
     def hasShuffleWrite: Boolean = shuffleWriteBytes > 0
-    def hasBytesSpilled: Boolean = memoryBytesSpilled > 0 && diskBytesSpilled
> 0
+    def hasBytesSpilled: Boolean = memoryBytesSpilled > 0 || diskBytesSpilled > 0
   }
 
   /**


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message