nemo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-nemo] hy00nc commented on issue #213: [NEMO-383] Implement DirectByteBufferOutputStream for Off-heap SerializedMemoryStore
Date Sun, 14 Apr 2019 10:09:14 GMT
hy00nc commented on issue #213: [NEMO-383] Implement DirectByteBufferOutputStream for Off-heap
SerializedMemoryStore
URL: https://github.com/apache/incubator-nemo/pull/213#issuecomment-482940459
 
 
   > +1 for adding getByteBuffer(). However, I think there would be some cases where toByteArray()
is still necessary. How about implementing both methods and add comments for toByteArray()
that it can degrade the performance due to the array copies? 2019년 4월 14일 (일) 오후
4:56, Taegeon Um <notifications@github.com>님이 작성:
   > […](#)
   > ***@***.**** commented on this pull request. ------------------------------ In common/src/main/java/org/apache/nemo/common/DirectByteBufferOutputStream.java
<[#213 (comment)](https://github.com/apache/incubator-nemo/pull/213#discussion_r275144557)>:
> + temp.flip(); + byteToWrite = temp.remaining(); + temp.get(byteArray, start, byteToWrite);
+ start += byteToWrite; + } + // The limit of the last buffer has to be set to the capacity
for re-write. + lastBuf.limit(lastBuf.capacity()); + + return byteArray; + } + + /** + * Closes
this output stream and releases resources. + */ + @override + public void close() { Avro also
implemented ByteBufferOutputStream, which returns the list of ByteBuffer. https://avro.apache.org/docs/1.5.4/api/java/org/apache/avro/util/ByteBufferOutputStream.html
— You are receiving this because you were mentioned. Reply to this email directly, view
it on GitHub <[#213 (comment)](https://github.com/apache/incubator-nemo/pull/213#discussion_r275144557)>,
or mute the thread <https://github.com/notifications/unsubscribe-auth/AETjDIpO-5R5WKCV4dWdKuoHFGycy8qBks5vgt8lgaJpZM4cuZ5J>
.
   
   I think creating `getByteBuffer()` is a nice idea too. I will add comments regarding the
performance issue of `toByteArray()`. And just like @taegeonum mentioned, I assume `close()`
method should do nothing for this output stream for now as long as there is a usage for `toByteArray()`.
In Nemo, I found the usage of obtaining the byte array after closing the `wrappedStream` which
includes the output stream. (though the comments say that closing the `wrapperStream` is dependent
on the property of `DirectByteArrayOutputStream`.)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message