hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <>
Subject [jira] [Commented] (HIVE-18825) Define ValidTxnList before starting query optimization
Date Wed, 21 Mar 2018 22:44:00 GMT


Jesus Camacho Rodriguez commented on HIVE-18825:

[~ekoifman], the code that accesses the valid txn list was not there when I created the patch,
it was introduced in HIVE-18751, hence I did not realize there was any dependency at that
level. I have made changes to avoid clearing the txn list, the rest should work as expected.

> Define ValidTxnList before starting query optimization
> ------------------------------------------------------
>                 Key: HIVE-18825
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Transactions
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>         Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch, HIVE-18825.03.patch, HIVE-18825.04.patch,
HIVE-18825.05.patch, HIVE-18825.06.patch, HIVE-18825.patch
> Consider a set of tables used by a materialized view where inserts happened after the
materialization was created. To compute incremental view maintenance, we need to be able
to filter only new rows from those base tables. That can be done by inserting a filter
operator with condition e.g. {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId
NOT IN(<open txns>)}} on top of the MVs query definition and triggering the rewriting
(which should in turn produce a partial rewriting). However, to do that, we need to have a
value for {{ValidTxnList}} during query compilation so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There should
not be any visible changes for end user.

This message was sent by Atlassian JIRA

View raw message