arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ARROW-5100) [JS] Writer swaps byte order if buffers share the same underlying ArrayBuffer
Date Wed, 03 Apr 2019 06:59:00 GMT
Paul Taylor created ARROW-5100:
----------------------------------

             Summary: [JS] Writer swaps byte order if buffers share the same underlying ArrayBuffer
                 Key: ARROW-5100
                 URL: https://issues.apache.org/jira/browse/ARROW-5100
             Project: Apache Arrow
          Issue Type: Bug
          Components: JavaScript
    Affects Versions: 0.13.0
            Reporter: Paul Taylor
            Assignee: Paul Taylor
             Fix For: 0.14.0


We collapse contiguous Uint8Arrays that share the same underlying ArrayBuffer and have overlapping
byte ranges. This was done to maintain true zero-copy behavior when using certain node core
streams that use a buffer pool internally, and could write chunks of the same logical Arrow
Message at out-of-order byte offsets in the pool.

Unfortunately this can also lead to a bug where, in rare cases, buffers are swapped while
writing Arrow Messages too. We could have a flag to indicate whether we think collapsing out-of-order
same-buffer chunks is safe, but I'm not sure if we can always know that, so I'd prefer to
take it out and incur the copy cost.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message