nemo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-nemo] johnyangk commented on a change in pull request #222: [NEMO-350] Implement Off-heap SerializedMemoryStore & [NEMO-384] Implement DirectByteBufferInputStream for Off-heap SerializedMemoryStore
Date Wed, 19 Jun 2019 07:22:55 GMT
johnyangk commented on a change in pull request #222: [NEMO-350] Implement Off-heap SerializedMemoryStore
& [NEMO-384] Implement DirectByteBufferInputStream for Off-heap SerializedMemoryStore
URL: https://github.com/apache/incubator-nemo/pull/222#discussion_r295136836
 
 

 ##########
 File path: common/src/main/java/org/apache/nemo/common/DirectByteBufferOutputStream.java
 ##########
 @@ -140,38 +140,45 @@ public void write(final byte[] b, final int off, final int len) {
     final int arraySize = pageSize * (dataList.size() - 1) + lastBuf.position();
     final byte[] byteArray = new byte[arraySize];
     int start = 0;
-    int byteToWrite;
 
-    for (final ByteBuffer temp : dataList) {
+    for (final ByteBuffer buffer : dataList) {
       // ByteBuffer has to be shifted to read mode by calling ByteBuffer.flip(),
       // which sets limit to the current position and sets the position to 0.
       // Note that capacity remains unchanged.
-      temp.flip();
-      byteToWrite = temp.remaining();
-      temp.get(byteArray, start, byteToWrite);
+      final ByteBuffer dupBuffer = buffer.duplicate();
+      dupBuffer.flip();
+      final int byteToWrite = dupBuffer.remaining();
+      dupBuffer.get(byteArray, start, byteToWrite);
 
 Review comment:
   I suppose here the contents of the off-heap bytes are copied to an on-heap array (i.e.,
bytesArray). If used in the execution code path, this would reduce the benefits of the off-heap
data management introduced in this PR.
   
   But, I believe `toByteArray()` is being used only in `DirectByteBufferOutputStreamTest`.
This led me to think that this method is intended to be used only in tests. If so, can you
rename the method make this intention explicit? It'd be good to also remove `public`.
   
   e.g., 
   ```java
   // USED BY TESTS ONLY
   @VisibleForTesting
   byte[] buildByteArrayForTests() {
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message