arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wes McKinney (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ARROW-245) [Format] Clarify Arrow's relationship with big endian platforms
Date Thu, 04 Aug 2016 20:19:20 GMT

    [ https://issues.apache.org/jira/browse/ARROW-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408425#comment-15408425
] 

Wes McKinney commented on ARROW-245:
------------------------------------

I'd be in favor of adding an endianness field to the RecordBatch flatbuffers table (default:
LITTLE)

https://github.com/apache/arrow/blob/master/format/Message.fbs#L140

Having a per-field endianness seems like unnecessary complexity. For now, on read we can validate
that the message matches our native endianness. 

> [Format] Clarify Arrow's relationship with big endian platforms
> ---------------------------------------------------------------
>
>                 Key: ARROW-245
>                 URL: https://issues.apache.org/jira/browse/ARROW-245
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Format
>            Reporter: Wes McKinney
>
> Per August 2016 mailing list question re: big endian platforms, we have in the format
document:
> https://github.com/apache/arrow/blob/master/format/Layout.md#byte-order-endianness
> We should clarify that this does not mean that Arrow cannot be used on big endian platforms,
but rather that the canonical or "in-flight" memory representation (for IPC or memory sharing
of any kind) is little-endian, so big endian systems would need to byte swap big endian integers
if they intend to expose memory to any other system using Arrow. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message