hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18839) Implement incremental rebuild for materialized views (only insert operations in source tables)
Date Mon, 02 Apr 2018 23:39:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423289#comment-16423289
] 

Jesus Camacho Rodriguez commented on HIVE-18839:
------------------------------------------------

[~ashutoshc], could you take a look at this one?
https://reviews.apache.org/r/66369/

Thanks

> Implement incremental rebuild for materialized views (only insert operations in source
tables)
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-18839
>                 URL: https://issues.apache.org/jira/browse/HIVE-18839
>             Project: Hive
>          Issue Type: Improvement
>          Components: Materialized views
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>              Labels: TODOC3.0
>         Attachments: HIVE-18839.patch
>
>
> Implementation will follow current code path for full rebuild. 
> When the MV query plan is retrieved, if the MV contents are outdated because there were
insert operations in the source tables, we will introduce a filter with a condition based
on stored value of ValidWriteIdLists. For instance, {{WRITE_ID < high_txn_id AND WRITE_ID
NOT IN (x, y, ...)}}. Then the rewriting will do the rest of the work by creating a partial
rewriting, where the contents of the MV are read as well as the new contents from the source
tables.
> This mechanism will not work only for ALTER MV... REBUILD, but also for user queries
which will be able to benefit from using outdated MVs to compute part of the needed results.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message