drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From paul-rogers <...@git.apache.org>
Subject [GitHub] drill pull request #1058: DRILL-6002: Avoid memory copy from direct buffer t...
Date Thu, 21 Dec 2017 01:46:13 GMT
Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1058#discussion_r158180527
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorSerializer.java
---
    @@ -62,27 +72,65 @@ public Writer write(VectorAccessible va) throws IOException {
     
         @SuppressWarnings("resource")
         public Writer write(VectorAccessible va, SelectionVector2 sv2) throws IOException
{
    +      checkNotNull(va);
           WritableBatch batch = WritableBatch.getBatchNoHVWrap(
               va.getRecordCount(), va, sv2 != null);
           return write(batch, sv2);
         }
     
         public Writer write(WritableBatch batch, SelectionVector2 sv2) throws IOException
{
    -      VectorAccessibleSerializable vas;
    -      if (sv2 == null) {
    -        vas = new VectorAccessibleSerializable(batch, allocator);
    -      } else {
    -        vas = new VectorAccessibleSerializable(batch, sv2, allocator);
    -      }
    -      if (retain) {
    -        vas.writeToStreamAndRetain(stream);
    -      } else {
    -        vas.writeToStream(stream);
    +      return write(batch, sv2, false);
    +    }
    +
    +    public Writer write(WritableBatch batch, SelectionVector2 sv2, boolean retain) throws
IOException {
    +      checkNotNull(batch);
    +      checkNotNull(channel);
    +      final Timer.Context timerContext = metrics.timer(WRITER_TIMER).time();
    +
    +      final DrillBuf[] incomingBuffers = batch.getBuffers();
    +      final UserBitShared.RecordBatchDef batchDef = batch.getDef();
    +
    +      try {
    +        /* Write the metadata to the file */
    +        batchDef.writeDelimitedTo(output);
    +
    +
    +        /* If we have a selection vector, dump it to file first */
    +        if (sv2 != null) {
    +          final int dataLength = sv2.getCount() * SelectionVector2.RECORD_SIZE;
    +          channel.write(sv2.getBuffer(false).nioBuffer(0, dataLength));
    +        }
    +
    +        /* Dump the array of ByteBuf's associated with the value vectors */
    +        for (DrillBuf buf : incomingBuffers) {
    +          /* dump the buffer into the OutputStream */
    +          channel.write(buf.nioBuffer());
    --- End diff --
    
    I believe Drill uses the `writerIndex` of `AbstractByteBuf` to indicate the end of the
written portion. So, we want to write that number of bytes. If that's what this code does,
we're good.


---

Mime
View raw message