atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vimal Sharma <visha...@hortonworks.com>
Subject Re: Review Request 52077: Column level lineage in Hive
Date Thu, 22 Sep 2016 11:48:34 GMT


> On Sept. 21, 2016, 8:58 p.m., Suma Shivaprasad wrote:
> > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/ColumnLineageUtils.java,
line 97
> > <https://reviews.apache.org/r/52077/diff/2/?file=1505617#file1505617line97>
> >
> >     why is column qualifiedName different from the convention we are using for hive_column
instances which are referred to from the table. Why is clustername removed?

Cluster information is not available in Lineage information provided by Hive. Further, qualifiedName
used in this patch is used only while setting column lineage and is not used for communication
with rest of Atlas codebase.


- Vimal


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52077/#review149896
-----------------------------------------------------------


On Sept. 20, 2016, 9:07 a.m., Vimal Sharma wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52077/
> -----------------------------------------------------------
> 
> (Updated Sept. 20, 2016, 9:07 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-247
>     https://issues.apache.org/jira/browse/ATLAS-247
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> After a CTAS query, lineage relationship between source columns and destination column
can be captured. This information can be used to create a column lineage process.
> 
> 
> Diffs
> -----
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/ColumnLineageUtils.java
PRE-CREATION 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java a3464a0 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataModelGenerator.java
45f0bc9 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataTypes.java e094cb6

>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java a5838b4

> 
> Diff: https://reviews.apache.org/r/52077/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Vimal Sharma
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message