drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6002) Avoid memory copy from direct buffer to heap while spilling to local disk
Date Sat, 02 Dec 2017 00:27:12 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275250#comment-16275250
] 

ASF GitHub Bot commented on DRILL-6002:
---------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1058#discussion_r154478390
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/cache/TestBatchSerialization.java
---
    @@ -117,11 +117,12 @@ public void testNullableType(MinorType type) throws IOException
{
       private void verifySerialize(SingleRowSet rowSet, SingleRowSet expected) throws IOException
{
     
         File dir = OperatorFixture.getTempDir("serial");
    -    File outFile = new File(dir, "serialze.dat");
    -    try (OutputStream out = new BufferedOutputStream(new FileOutputStream(outFile)))
{
    -      VectorSerializer.writer(fixture.allocator(), out)
    -        .write(rowSet.container(), rowSet.getSv2());
    -    }
    +    VectorSerializer.writer(new SpillSet(dir.getAbsolutePath()),  "serialize.dat")
    --- End diff --
    
    The `File dir` line should change to use Tim's new directory watcher feature. 


> Avoid memory copy from direct buffer to heap while spilling to local disk
> -------------------------------------------------------------------------
>
>                 Key: DRILL-6002
>                 URL: https://issues.apache.org/jira/browse/DRILL-6002
>             Project: Apache Drill
>          Issue Type: Improvement
>            Reporter: Vlad Rozov
>            Assignee: Vlad Rozov
>
> When spilling to a local disk or to any file system that supports WritableByteChannel
it is preferable to avoid copy from off-heap to java heap as WritableByteChannel can work
directly with the off-heap memory.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message