nemo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-nemo] hy00nc commented on a change in pull request #223: [NEMO-388] Off-heap memory management (reuse ByteBuffer)
Date Thu, 25 Jul 2019 02:45:04 GMT
hy00nc commented on a change in pull request #223: [NEMO-388] Off-heap memory management (reuse
ByteBuffer)
URL: https://github.com/apache/incubator-nemo/pull/223#discussion_r307094944
 
 

 ##########
 File path: runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/partition/SerializedPartition.java
 ##########
 @@ -226,4 +236,11 @@ public int getLength() throws IOException {
   public boolean isOffheap() {
     return offheap;
   }
+
+  /**
+   * Releases the off-heap memory that this SerializedPartition holds.
+   */
+  public void release() {
+    bytesOutputStream.release();
 
 Review comment:
   Actually, `release()` operates on the **data**, although it seems like it works on the
stream. The stream here uses a list of ByteBuffers wrapped as MemoryChunk, which is the data
itself. The `dataList` in SerializedPartition shares the list of ByteBuffers of the output
stream, so if this is called upon commit(), it will return the 'uncleared' MemoryChunks back
to the pool, causing correctness problem. Maybe I could add some comments on this?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message