spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joshro...@apache.org
Subject spark git commit: [SPARK-17056][CORE] Fix a wrong assert regarding unroll memory in MemoryStore
Date Tue, 27 Sep 2016 23:00:52 GMT
Repository: spark
Updated Branches:
  refs/heads/master 2f84a6866 -> e7bce9e18


[SPARK-17056][CORE] Fix a wrong assert regarding unroll memory in MemoryStore

## What changes were proposed in this pull request?

There is an assert in MemoryStore's putIteratorAsValues method which is used to check if unroll
memory is not released too much. This assert looks wrong.

## How was this patch tested?

Jenkins tests.

Author: Liang-Chi Hsieh <simonh@tw.ibm.com>

Closes #14642 from viirya/fix-unroll-memory.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e7bce9e1
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e7bce9e1
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e7bce9e1

Branch: refs/heads/master
Commit: e7bce9e1876de6ee975ccc89351db58119674aef
Parents: 2f84a68
Author: Liang-Chi Hsieh <simonh@tw.ibm.com>
Authored: Tue Sep 27 16:00:39 2016 -0700
Committer: Josh Rosen <joshrosen@databricks.com>
Committed: Tue Sep 27 16:00:39 2016 -0700

----------------------------------------------------------------------
 .../main/scala/org/apache/spark/storage/memory/MemoryStore.scala   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/e7bce9e1/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala b/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala
index 205d469..095d324 100644
--- a/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala
+++ b/core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala
@@ -273,7 +273,7 @@ private[spark] class MemoryStore(
           blockId, Utils.bytesToString(size), Utils.bytesToString(maxMemory - blocksMemoryUsed)))
         Right(size)
       } else {
-        assert(currentUnrollMemoryForThisTask >= currentUnrollMemoryForThisTask,
+        assert(currentUnrollMemoryForThisTask >= unrollMemoryUsedByThisBlock,
           "released too much unroll memory")
         Left(new PartiallyUnrolledIterator(
           this,


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message