hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14249) Add simple materialized views with manual rebuilds
Date Wed, 20 Jul 2016 12:37:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385776#comment-15385776
] 

Jesus Camacho Rodriguez commented on HIVE-14249:
------------------------------------------------

[~alangates], I have created a pull request in https://github.com/apache/hive/pull/91 .

I have rebased your initial patch so it would apply cleanly to master. Initial patch contained
a lot of code already, in particular support for CREATE MATERIALIZED VIEW mv ..., ALTER MATERIALIZED
VIEW mv REBUILD, and DROP MATERIALIZED VIEW mv. It also contained authorization bits for creating/accessing
the materialized views. In addition to positive/negative tests for these cases.

I created a second commit that extends the original patch with some useful features. In particular,
being able to add properties to the MV, use a custom StorageHandler, and specifying a custom
location to store the data: all these features will be useful if we want to integrate MVs
with other external systems e.g. Druid. In addition, I enabled Calcite optimization of the
MV query, as before we were bypassing the optimizer. Finally, I extended existing tests and
added new tests. Could you review this second commit? Thanks

I think those two commits have the initial blocks for MVs in place. One of the remaining features
that I wanted to add was the support of partitioning for MVs, as I think it would be quite
useful for performance and follow-up maintenance implementation; however, I checked the code
in SemanticAnalyzer, etc. a bit and I think this is not straightforward. If you have a clear
idea in mind on the bits that we need to implement to support partitioning in MVs, please
let me know.

Once the patch goes in (the support for partitioning is not needed), I can create a follow-up
issue to start the integration with Calcite and its views service, hence starting to experiment
with its query rewriting capabilities using materialized views.

> Add simple materialized views with manual rebuilds
> --------------------------------------------------
>
>                 Key: HIVE-14249
>                 URL: https://issues.apache.org/jira/browse/HIVE-14249
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Parser, Views
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: HIVE-10459.2.patch
>
>
> This patch is a start at implementing simple views. It doesn't have enough testing yet
(e.g. there's no negative testing). And I know it fails in the partitioned case. I suspect
things like security and locking don't work properly yet either. But I'm posting it as a starting
point.
> In this initial patch I'm just handling simple materialized views with manual rebuilds.
In later JIRAs we can add features such as allowing the optimizer to rewrite queries to use
materialized views rather than tables named in the queries, giving the optimizer the ability
to determine when a materialized view is stale, etc.
> Also, I didn't rebase this patch against trunk after the migration from svn->git so
it may not apply cleanly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message