arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wes McKinney (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ARROW-56) Format: Specify LSB bit ordering in bit arrays
Date Thu, 10 Mar 2016 00:36:40 GMT
Wes McKinney created ARROW-56:
---------------------------------

             Summary: Format: Specify LSB bit ordering in bit arrays
                 Key: ARROW-56
                 URL: https://issues.apache.org/jira/browse/ARROW-56
             Project: Apache Arrow
          Issue Type: New Feature
          Components: Format
            Reporter: Wes McKinney


In https://github.com/apache/arrow/blob/master/format/Layout.md, we describe bitmaps for nulls
(presumably same encoding for boolean data) and provide the C code for accessing a bit:

{{bitmask[j / 8] & (1 << (j % 8))}}

Implicitly this is LSB bit-endianness (https://en.wikipedia.org/wiki/Bit_numbering) -- see
the discussion of bitpacking in the Parquet format: https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3

If this is what we want, we should make this aspect of the format explicit. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message