hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenlei Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1073) CREATE VIEW followup: track view dependency information in metastore
Date Tue, 24 Nov 2015 19:28:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025155#comment-15025155
] 

Wenlei Xie commented on HIVE-1073:
----------------------------------

There are two things I would like to discuss first:

1. To track the view dependency, I will add a new class {{MTableDependency}} (name TBD) which
contains {{srcTbl}} and {{dstTbl}}. A corresponding table with 2 columns ({{srcTblId}} and
{{dstTblId}}) will be added into the JDO database. This table / data structure tracks the
view dependency. 

We will use the JDOQL to fetch the dependent/dependee tables for a given table.

2. When a view is created/altered, the Metastore Server needs to know the dependent tables
of the view. I don’t want to make the {{hive-metastore}} package depends on {{hive-exec}}
package (which contains the parsing code). Thus the Hive client have to send the dependent
tables of view to the server (for example, by adding a {{dependentTables}} filed in the Table
class). 

However, this means the Metastore server cannot verify whether the dependent table reported
by the client side is consistent with the SQL query creating the view. This also means an
old version of Hive Client cannot work with the a new version of Metastore Server that tracks
view dependency. 


> CREATE VIEW followup:  track view dependency information in metastore
> ---------------------------------------------------------------------
>
>                 Key: HIVE-1073
>                 URL: https://issues.apache.org/jira/browse/HIVE-1073
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore, Views
>    Affects Versions: 0.6.0
>            Reporter: John Sichi
>            Assignee: Wenlei Xie
>
> Add a generic mechanism for recording the fact that one object depends on another.  First
use case (to be implemented as part of this task) would be views depending on tables or other
views, but in the future we can also use this for views depending on persistent functions,
functions depending on other functions, etc.
> This involves metastore modeling, QL analysis for deriving and recording the dependencies
(Ashish says something may already be available from the lineage work), and CLI support for
browsing dependencies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message