hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghotham Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1131) Add column lineage information to the pre execution hooks
Date Thu, 18 Feb 2010 23:25:28 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835487#action_12835487

Raghotham Murthy commented on HIVE-1131:

Went over code with Ashish. A few things: 

1. The hash<key1, hash<key2, value>> paradigm can be changed to hash<pair<key1,key2>,
value>. That will reduce the amount of code needed. For example, there is no need for special
iterator and item classes.
2. Code which records visits to nodes can be removed
3. PreOrderWalker.java does not have any change

> Add column lineage information to the pre execution hooks
> ---------------------------------------------------------
>                 Key: HIVE-1131
>                 URL: https://issues.apache.org/jira/browse/HIVE-1131
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Ashish Thusoo
>            Assignee: Ashish Thusoo
>         Attachments: HIVE-1131.patch
> We need a mechanism to pass the lineage information of the various columns of a table
to a pre execution hook so that applications can use that for:
> - auditing
> - dependency checking
> and many other applications.
> The proposal is to expose this through a bunch of classes to the pre execution hook interface
to the clients and put in the necessary transformation logic in the optimizer to generate
this information.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message