arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ARROW-41) C++: Convert table to std::vector of Struct arrays
Date Sun, 27 Mar 2016 01:19:25 GMT

    [ https://issues.apache.org/jira/browse/ARROW-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213281#comment-15213281
] 

Kai Zheng commented on ARROW-41:
--------------------------------

Thanks [~wesmckinn] for reporting this. I have some questions, would you help with? Thanks.
bq. Convert table to std::vector of {{Struct arrays}}
I don't find any real codes for {{struct}} type and {{struct array}}. Where am I lost?
bq. While tables and struct type columns are semantically equivalent (and tables can be embedded
in other tables using struct types), the memory layout of a table may not be strictly contiguous.
Good to know the difference. The conversion sounds like, table => struct; table->column
=> struct->field with column name being the field name; the resultant struct will be
strictly contiguous. Right?
bq. For the purposes of putting data on the wire / in shared memory, it may be useful to offer
a conversion function to "structify" an in-memory logical Arrow table. 
Not quite getting this. Do you mean arrow table isn't suitable for the wire or to be remained
in shared memory? I looked at the codes, a table mainly consists of columns and columns are
just chunked arrays, which looks like to me rather compact and not bad for the mentioned purpose.

> C++: Convert table to std::vector of Struct arrays
> --------------------------------------------------
>
>                 Key: ARROW-41
>                 URL: https://issues.apache.org/jira/browse/ARROW-41
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++
>            Reporter: Wes McKinney
>
> This may require memory allocation depending on the chunking of the table columns. 
> While tables and struct type columns are semantically equivalent (and tables can be embedded
in other tables using struct types), the memory layout of a table may not be strictly contiguous.
For the purposes of putting data on the wire / in shared memory, it may be useful to offer
a conversion function to "structify" an in-memory logical Arrow table. See ARROW-24



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message