arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Le Dem (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (ARROW-399) [Java] ListVector.loadFieldBuffers ignores the ArrowFieldNode length metadata
Date Thu, 01 Dec 2016 20:05:58 GMT

     [ https://issues.apache.org/jira/browse/ARROW-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Julien Le Dem reassigned ARROW-399:
-----------------------------------

    Assignee: Julien Le Dem

> [Java] ListVector.loadFieldBuffers ignores the ArrowFieldNode length metadata
> -----------------------------------------------------------------------------
>
>                 Key: ARROW-399
>                 URL: https://issues.apache.org/jira/browse/ARROW-399
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Java - Vectors
>            Reporter: Wes McKinney
>            Assignee: Julien Le Dem
>            Priority: Blocker
>         Attachments: list_error.json
>
>
> Discovered this during integration testing. Because Arrow-C++ writes buffers padded to
64 bytes, they may appear larger to the Java library than they need to be. In ListVector.loadFieldBuffers,
the ArrowFieldNode is never used:
> {code:language=java}
>   @Override
>   public void loadFieldBuffers(ArrowFieldNode fieldNode, List<ArrowBuf> ownBuffers)
{
>     BaseDataValueVector.load(getFieldInnerVectors(), ownBuffers);
>   }
> {code}
> The value count of the resulting ListVector is thus inferred from the size of the offsets
buffer. In the case of a length-7 vector in C++, the size of the offsets buffer is exactly
64 bytes (padding for SIMD) -- Java infers from 64 bytes that the value count is 15 (64 /
4 - 1), and the integration test fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message