hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mustafa Iman (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22555) Upgrade ORC version to 1.5.8
Date Wed, 27 Nov 2019 18:22:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983792#comment-16983792
] 

Mustafa Iman commented on HIVE-22555:
-------------------------------------

Failing tests while upgrading to ORC 1.5.7 version:
{code:java}
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=224)
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testMemoryManagementV11[0] (batchId=326)
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testMemoryManagementV11[1] (batchId=326)
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testMemoryManagementV12[0] (batchId=326)
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testMemoryManagementV12[1] (batchId=326)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderNewBaseAndDelta (batchId=327)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderOldBaseAndDelta (batchId=327){code}
All these tests fail because they are relying on a custom MemoryManager. They depend on number
of added rows that is reported by MemoryManagerImpl in ORC. However, ORC 1.5.7 does not report
number of added rows. Instead it reports scaling factor when it changes. The tests need to
conform.

 

All the failures are due to 1.5.7 changes. Once they are resolved, 1.5.8 should work without
further changes.

> Upgrade ORC version to 1.5.8
> ----------------------------
>
>                 Key: HIVE-22555
>                 URL: https://issues.apache.org/jira/browse/HIVE-22555
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 4.0.0
>            Reporter: Mustafa Iman
>            Assignee: Mustafa Iman
>            Priority: Major
>
> Hive currently depends on ORC 1.5.6. We need 1.5.8 upgrade for https://issues.apache.org/jira/browse/HIVE-22499
> ORC-1.5.7 includes https://issues.apache.org/jira/browse/ORC-361 . It causes some tests
overriding MemoryManager to fail. These need to be addressed while upgrading.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message