hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13285) Orc concatenation may drop old files from moving to final path
Date Wed, 16 Mar 2016 21:01:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198151#comment-15198151
] 

Prasanth Jayachandran commented on HIVE-13285:
----------------------------------------------

TestMiniTezCliDriver failures are due to initialization failure. I ran all the tests locally
and they all passed. Spark test failures are happening in other jiras as well. 

> Orc concatenation may drop old files from moving to final path
> --------------------------------------------------------------
>
>                 Key: HIVE-13285
>                 URL: https://issues.apache.org/jira/browse/HIVE-13285
>             Project: Hive
>          Issue Type: Bug
>          Components: ORC
>    Affects Versions: 0.14.0, 1.0.0, 1.2.0, 1.1.0, 1.3.0, 2.0.0, 2.1.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Critical
>         Attachments: HIVE-13285.1.patch, HIVE-13285.2.patch, HIVE-13285.3.patch
>
>
> ORC concatenation uses combine hive input format for merging files. Under specific case
where all files within a combine split are incompatible for merge (old files without stripe
statistics) then these files are added to incompatible file set. But this file set is not
processed as closeOp() will not be called (no output file writer will exist which will skip
super.closeOp()). As a result, these incompatible files are not moved to final path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message