hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saket Saurabh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
Date Tue, 09 Aug 2016 22:47:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414383#comment-15414383
] 

Saket Saurabh commented on HIVE-14035:
--------------------------------------

[~sershe] Thanks for the comments on RB. I am working on fixing those. No, the last run for
patch 13 did not have split-update enabled by default. There are many tests that assert on
number of files and directory layout that would anyway fail in PTest if we run those tests
w/o modification. However, excluding those assert failures, when I ran these locally, the
only other failures were NegativeArrayIndexException & IndexOutOfBoundException caused
by HIVE-14448 and not related to this patch. However, I have created a subclass TestTxnCommands3
that should ideally mimic this behavior with split-update enabled by default for a large number
of ACID scenarios.

> Enable predicate pushdown to delta files created by ACID Transactions
> ---------------------------------------------------------------------
>
>                 Key: HIVE-14035
>                 URL: https://issues.apache.org/jira/browse/HIVE-14035
>             Project: Hive
>          Issue Type: New Feature
>          Components: Transactions
>            Reporter: Saket Saurabh
>            Assignee: Saket Saurabh
>         Attachments: HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch, HIVE-14035.05.patch,
HIVE-14035.06.patch, HIVE-14035.07.patch, HIVE-14035.08.patch, HIVE-14035.09.patch, HIVE-14035.10.patch,
HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, HIVE-14035.patch
>
>
> In current Hive version, delta files created by ACID transactions do not allow predicate
pushdown if they contain any update/delete events. This is done to preserve correctness when
following a multi-version approach during event collapsing, where an update event overwrites
an existing insert event. 
> This JIRA proposes to split an update event into a combination of a delete event followed
by a new insert event, that can enable predicate push down to all delta files without breaking
correctness. To support backward compatibility for this feature, this JIRA also proposes to
add some sort of versioning to ACID that can allow different versions of ACID transactions
to co-exist together.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message