hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10591) Support limited integer type promotion in ORC
Date Sun, 10 May 2015 00:09:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536963#comment-14536963
] 

Prasanth Jayachandran commented on HIVE-10591:
----------------------------------------------

The encryption_insert_partition_static.q test ran successfully when I ran it locally. Also
the test diff shows file size differences which has nothing to do with this patch as the changes
of this patch is read side only.

> Support limited integer type promotion in ORC
> ---------------------------------------------
>
>                 Key: HIVE-10591
>                 URL: https://issues.apache.org/jira/browse/HIVE-10591
>             Project: Hive
>          Issue Type: New Feature
>    Affects Versions: 1.3.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>             Fix For: 1.3.0
>
>         Attachments: HIVE-10591.1.patch, HIVE-10591.2.patch, HIVE-10591.2.patch, HIVE-10591.3.patch,
HIVE-10591.3.patch, HIVE-10591.3.patch
>
>
> ORC currently does not support schema-on-read. If we alter an ORC table with 'int' type
to 'bigint' and if we query the altered table ClassCastException will be thrown as the schema
on read from table descriptor will expect LongWritable whereas ORC will return IntWritable
based on file schema stored within ORC file. OrcSerde currently doesn't do any type conversions
or type promotions for performance reasons in inner loop. Since smallints, ints and bigints
are stored in the same way in ORC, it will be possible be allow such type promotions without
hurting performance. Following type promotions can be supported without any casting
> smallint -> int
> smallint -> bigint
> int -> bigint
> Tinyint promotion is not possible without casting as tinyints are stored using RLE byte
writer whereas smallints, ints and bigints are stored using RLE integer writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message