hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-9067) OrcFileMergeOperator may create merge file that does not match properties of input files
Date Fri, 12 Dec 2014 01:43:14 GMT

     [ https://issues.apache.org/jira/browse/HIVE-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-9067:
----------------------------------------
    Attachment: HIVE-9067.2.patch

Added fix for HIVE-9080 here as both are relevant (in the same class). The new changes is
to address updation of file statistics from stripe statistics properly. [~sershe] Can you
take a look at the new changes? Updating file statistics in the presence of complex column
types (struct, union, list, map) is addressed with this new change.

> OrcFileMergeOperator may create merge file that does not match properties of input files
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-9067
>                 URL: https://issues.apache.org/jira/browse/HIVE-9067
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.14.0, 0.15.0, 0.14.1
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Minor
>              Labels: Orc
>         Attachments: HIVE-9067.1.patch, HIVE-9067.2.patch
>
>
> OrcFileMergeOperator creates a new ORC file and appends the stripes from smaller orc
file. This new ORC file creation should retain the same configuration as the small ORC files.
Currently it does not set the orc row index stride and file version. Also merging of stripe
statistics to file statistics was incorrect leading to issues like in HIVE-9080



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message