spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject spark git commit: [SPARK-15246][SPARK-4452][CORE] Fix code style and improve volatile for
Date Wed, 11 May 2016 05:30:47 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-2.0 1b446a461 -> d9288b804


[SPARK-15246][SPARK-4452][CORE] Fix code style and improve volatile for

## What changes were proposed in this pull request?
1. Fix code style
2. remove volatile of elementsRead method because there is only one thread to use it.
3. avoid volatile of _elementsRead because Collection increases number of  _elementsRead when
it insert a element. It is very expensive. So we can avoid it.

After this PR, I will push another PR for branch 1.6.
## How was this patch tested?
unit tests

Author: Lianhui Wang <lianhuiwang09@gmail.com>

Closes #13020 from lianhuiwang/SPARK-4452-hotfix.

(cherry picked from commit 9f0a642f842df4d46f809c83df5e8b1803ab89a2)
Signed-off-by: Reynold Xin <rxin@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d9288b80
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d9288b80
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d9288b80

Branch: refs/heads/branch-2.0
Commit: d9288b8040bc2f74b8cf9e0b9342da61277916a7
Parents: 1b446a4
Author: Lianhui Wang <lianhuiwang09@gmail.com>
Authored: Tue May 10 22:30:39 2016 -0700
Committer: Reynold Xin <rxin@databricks.com>
Committed: Tue May 10 22:30:44 2016 -0700

----------------------------------------------------------------------
 core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java | 2 +-
 .../main/scala/org/apache/spark/util/collection/Spillable.scala   | 3 +--
 2 files changed, 2 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/d9288b80/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java
----------------------------------------------------------------------
diff --git a/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java b/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java
index 89521c1..a05a79c 100644
--- a/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java
+++ b/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java
@@ -413,7 +413,7 @@ public class TaskMemoryManager {
   /**
    * Returns Tungsten memory mode
    */
-  public MemoryMode getTungstenMemoryMode(){
+  public MemoryMode getTungstenMemoryMode() {
     return tungstenMemoryMode;
   }
 }

http://git-wip-us.apache.org/repos/asf/spark/blob/d9288b80/core/src/main/scala/org/apache/spark/util/collection/Spillable.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/util/collection/Spillable.scala b/core/src/main/scala/org/apache/spark/util/collection/Spillable.scala
index aee6399..bdcbd22 100644
--- a/core/src/main/scala/org/apache/spark/util/collection/Spillable.scala
+++ b/core/src/main/scala/org/apache/spark/util/collection/Spillable.scala
@@ -41,7 +41,7 @@ private[spark] abstract class Spillable[C](taskMemoryManager: TaskMemoryManager)
   protected def forceSpill(): Boolean
 
   // Number of elements read from input since last spill
-  @volatile protected def elementsRead: Long = _elementsRead
+  protected def elementsRead: Long = _elementsRead
 
   // Called by subclasses every time a record is read
   // It's used for checking spilling frequency
@@ -112,7 +112,6 @@ private[spark] abstract class Spillable[C](taskMemoryManager: TaskMemoryManager)
       if (!isSpilled) {
         0L
       } else {
-        _elementsRead = 0
         val freeMemory = myMemoryThreshold - initialMemoryThreshold
         _memoryBytesSpilled += freeMemory
         releaseMemory()


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message