spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joshro...@apache.org
Subject spark git commit: [SPARK-12165][ADDENDUM] Fix outdated comments on unroll test
Date Thu, 10 Dec 2015 01:24:24 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.6 9bc6a27fd -> d86a88da6


[SPARK-12165][ADDENDUM] Fix outdated comments on unroll test

JoshRosen

Author: Andrew Or <andrew@databricks.com>

Closes #10229 from andrewor14/unroll-test-comments.

(cherry picked from commit 8770bd1213f9b1051dabde9c5424ae7b32143a44)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d86a88da
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d86a88da
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d86a88da

Branch: refs/heads/branch-1.6
Commit: d86a88da677041d3c4ab484ed6f4f152674091f0
Parents: 9bc6a27
Author: Andrew Or <andrew@databricks.com>
Authored: Wed Dec 9 17:24:04 2015 -0800
Committer: Josh Rosen <joshrosen@databricks.com>
Committed: Wed Dec 9 17:24:18 2015 -0800

----------------------------------------------------------------------
 .../apache/spark/memory/StaticMemoryManagerSuite.scala | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/d86a88da/core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala
----------------------------------------------------------------------
diff --git a/core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala b/core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala
index 6700b94..272253b 100644
--- a/core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala
+++ b/core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala
@@ -163,15 +163,20 @@ class StaticMemoryManagerSuite extends MemoryManagerSuite {
     assertEvictBlocksToFreeSpaceNotCalled(ms)
     assert(mm.storageMemoryUsed === 860L)
     // `spark.storage.unrollFraction` is 0.4, so the max unroll space is 400 bytes.
-    // Since we already occupy 60 bytes, we will try to evict only 400 - 60 = 340 bytes.
+    // As of this point, cache memory is 800 bytes and current unroll memory is 60 bytes.
+    // Requesting 240 more bytes of unroll memory will leave our total unroll memory at
+    // 300 bytes, still under the 400-byte limit. Therefore, all 240 bytes are granted.
     assert(mm.acquireUnrollMemory(dummyBlock, 240L, evictedBlocks))
-    assertEvictBlocksToFreeSpaceCalled(ms, 100L)
+    assertEvictBlocksToFreeSpaceCalled(ms, 100L) // 860 + 240 - 1000
     when(ms.currentUnrollMemory).thenReturn(300L) // 60 + 240
     assert(mm.storageMemoryUsed === 1000L)
     evictedBlocks.clear()
+    // We already have 300 bytes of unroll memory, so requesting 150 more will leave us
+    // above the 400-byte limit. Since there is not enough free memory, this request will
+    // fail even after evicting as much as we can (400 - 300 = 100 bytes).
     assert(!mm.acquireUnrollMemory(dummyBlock, 150L, evictedBlocks))
-    assertEvictBlocksToFreeSpaceCalled(ms, 100L) // 400 - 300
-    assert(mm.storageMemoryUsed === 900L) // 100 bytes were evicted
+    assertEvictBlocksToFreeSpaceCalled(ms, 100L)
+    assert(mm.storageMemoryUsed === 900L)
     // Release beyond what was acquired
     mm.releaseUnrollMemory(maxStorageMem)
     assert(mm.storageMemoryUsed === 0L)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message