hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14035) Enable predicate pushdown to delta files created by ACID Transactions
Date Sat, 13 Aug 2016 06:50:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419832#comment-15419832
] 

Lefty Leverenz commented on HIVE-14035:
---------------------------------------

[~saketj], I added a link to your design document (version 2) in the wiki's Design Docs. 
Perhaps the link will become out-of-date if you revise the doc again, but eventually the doc
should be reformatted as a wiki page.

* [Design Docs -- In Progress | https://cwiki.apache.org/confluence/display/Hive/DesignDocs#DesignDocs-InProgress]

> Enable predicate pushdown to delta files created by ACID Transactions
> ---------------------------------------------------------------------
>
>                 Key: HIVE-14035
>                 URL: https://issues.apache.org/jira/browse/HIVE-14035
>             Project: Hive
>          Issue Type: New Feature
>          Components: Transactions
>            Reporter: Saket Saurabh
>            Assignee: Saket Saurabh
>             Fix For: 2.2.0
>
>         Attachments: Design.Document.Improving ACID performance in Hive.01.docx, Design.Document.Improving
ACID performance in Hive.02.docx, HIVE-14035.02.patch, HIVE-14035.03.patch, HIVE-14035.04.patch,
HIVE-14035.05.patch, HIVE-14035.06.patch, HIVE-14035.07.patch, HIVE-14035.08.patch, HIVE-14035.09.patch,
HIVE-14035.10.patch, HIVE-14035.11.patch, HIVE-14035.12.patch, HIVE-14035.13.patch, HIVE-14035.14.patch,
HIVE-14035.15.patch, HIVE-14035.16.patch, HIVE-14035.17.patch, HIVE-14035.patch
>
>
> In current Hive version, delta files created by ACID transactions do not allow predicate
pushdown if they contain any update/delete events. This is done to preserve correctness when
following a multi-version approach during event collapsing, where an update event overwrites
an existing insert event. 
> This JIRA proposes to split an update event into a combination of a delete event followed
by a new insert event, that can enable predicate push down to all delta files without breaking
correctness. To support backward compatibility for this feature, this JIRA also proposes to
add some sort of versioning to ACID that can allow different versions of ACID transactions
to co-exist together.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message