arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng, Kai" <kai.zh...@intel.com>
Subject More questions about layout spec
Date Sun, 10 Apr 2016 02:55:06 GMT
Hi,

Looking at the layout spec, I have some more questions to complement to the previous ones
discussed.

About struct and union types:

1)      The order of the fields (how they are declared in order) seems to be important, as
it will affect how the data are laid out. For example, in union type, how to organize and
interpret field types, offsets and data arrays. Similar to struct type.

2)      There is no saying about how their schema is represented and how/where the schema
is attached. Should the layout also contain the schema info? In cpp codes, Table is implemented
of columns and with self-contained schema info.

About schema:
Nothing is mentioned about schema in the spec, no sure if it should be the nature part of
it. Without self-contained schema info, it won't be able to interpret and process the layout
data (like List, Struct, Table, Union and etc.) across machines and languages.

About non-goals:
The follow are listed as non-goals for the document but in fact they're going to be implemented.
Should we remove them?

1.       To specify standardized metadata or a data layout for RPC or transient file storage.

2.       Any "table" structure composed of named arrays each having their own type or any
other structure that composes arrays.

Regards,
Kai


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message