spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wenc...@apache.org
Subject spark git commit: [SPARK-10399][SPARK-23879][FOLLOWUP][CORE] Free unused off-heap memory in MemoryBlockSuite
Date Mon, 23 Apr 2018 02:45:30 GMT
Repository: spark
Updated Branches:
  refs/heads/master c48085aa9 -> c3a86faa5


[SPARK-10399][SPARK-23879][FOLLOWUP][CORE] Free unused off-heap memory in MemoryBlockSuite

## What changes were proposed in this pull request?

As viirya pointed out [here](https://github.com/apache/spark/pull/19222#discussion_r179910484),
this PR explicitly frees unused off-heap memory in `MemoryBlockSuite`

## How was this patch tested?

Existing UTs

Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com>

Closes #21117 from kiszk/SPARK-10399-free-offheap.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c3a86faa
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c3a86faa
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c3a86faa

Branch: refs/heads/master
Commit: c3a86faa53c9e49efd595802adc38a6d412ce681
Parents: c48085a
Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
Authored: Mon Apr 23 10:45:25 2018 +0800
Committer: Wenchen Fan <wenchen@databricks.com>
Committed: Mon Apr 23 10:45:25 2018 +0800

----------------------------------------------------------------------
 .../java/org/apache/spark/unsafe/memory/MemoryBlockSuite.java    | 4 ++++
 1 file changed, 4 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/c3a86faa/common/unsafe/src/test/java/org/apache/spark/unsafe/memory/MemoryBlockSuite.java
----------------------------------------------------------------------
diff --git a/common/unsafe/src/test/java/org/apache/spark/unsafe/memory/MemoryBlockSuite.java
b/common/unsafe/src/test/java/org/apache/spark/unsafe/memory/MemoryBlockSuite.java
index 5d5fdc1..ef5ff8e 100644
--- a/common/unsafe/src/test/java/org/apache/spark/unsafe/memory/MemoryBlockSuite.java
+++ b/common/unsafe/src/test/java/org/apache/spark/unsafe/memory/MemoryBlockSuite.java
@@ -120,6 +120,8 @@ public class MemoryBlockSuite {
     } catch (Exception expected) {
       Assert.assertThat(expected.getMessage(), containsString("should not be larger than"));
     }
+
+    memory.setPageNumber(MemoryBlock.NO_PAGE_NUMBER);
   }
 
   @Test
@@ -165,11 +167,13 @@ public class MemoryBlockSuite {
     int length = 56;
 
     check(memory, obj, offset, length);
+    memoryAllocator.free(memory);
 
     long address = Platform.allocateMemory(112);
     memory = new OffHeapMemoryBlock(address, length);
     obj = memory.getBaseObject();
     offset = memory.getBaseOffset();
     check(memory, obj, offset, length);
+    Platform.freeMemory(address);
   }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message