orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xndai <...@git.apache.org>
Subject [GitHub] orc pull request #245: ORC-161: Proposal for new decimal encodings and stati...
Date Thu, 12 Apr 2018 16:47:15 GMT
Github user xndai commented on a diff in the pull request:

    https://github.com/apache/orc/pull/245#discussion_r181149073
  
    --- Diff: site/_docs/encodings.md ---
    @@ -123,6 +127,41 @@ DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
                   | DATA            | No       | Unbounded base 128 varints
                   | SECONDARY       | No       | Unsigned Integer RLE v2
     
    +In ORC 2.0, DECIMAL and DECIMAL_V2 encodings are introduced and scale
    +stream is totally removed as all decimal values use the same scale.
    +There are two difference cases: precision<=18 and precision>18.
    +
    +### Decimal Encoding for precision <= 18
    +
    +When precision is no greater than 18, decimal values can be fully
    +represented by 64-bit signed integers which are stored in DATA stream
    +and use signed integer RLE.
    +
    +Encoding      | Stream Kind     | Optional | Contents
    +:------------ | :-------------- | :------- | :-------
    +DECIMAL       | PRESENT         | Yes      | Boolean RLE
    +              | DATA            | No       | Signed Integer RLE v1
    +DECIMAL_V2    | PRESENT         | Yes      | Boolean RLE
    +              | DATA            | No       | Signed Integer RLE v2
    --- End diff --
    
    I think we should keep RLE v1 as an option. The C++ writer currently does not support
RLE v2 (we are working on it). We don't want the new decimal writer to have dependency on
that.


---

Mime
View raw message