atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suma Shivaprasad <>
Subject Re: Review Request 45784: Hve Hook - Support tracking lineage for External Tables( Create/alter) , Load, import, export
Date Wed, 06 Apr 2016 17:44:27 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated April 6, 2016, 5:44 p.m.)

Review request for atlas.


Fixed review comments

Bugs: ATLAS-527

Repository: atlas


Added support to track lineage between HDFS Paths and hive tables  in 

a. LOAD( at table, partition level) - input is a HDFS path and output is table( even though
we dont create partition entities, we are still tracking the lineage at table level for partitions.
This could be an issue if there are large number of partition queries which is not being addressed
in this jira - . refer
b. IMPORT, EXPORT to and from hdfs paths - Refer
c. CREATE EXTERNAL TABLE - input is hdfs path and o/p is table
d. ALTER TABLE LOCATION for an external table - input is the new hdfs path and o/p is the

Also changed the ordering of model registration by sorting them by modifiedTime to ensure
they are registered in correct order

Diffs (updated)

  addons/hdfs-model/src/main/java/org/apache/atlas/fs/model/ 555d565

  addons/hdfs-model/src/main/scala/org/apache/atlas/fs/model/FSDataModel.scala c964f73 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/ 3a802d7

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/ 68e32ff 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/ e17afb8 
  addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/ 5665856

  client/src/main/java/org/apache/atlas/ c3b4ba9 
  repository/src/main/java/org/apache/atlas/services/ 0a04c5f 
  repository/src/main/java/org/apache/atlas/services/ 430bb6b 



Added tests in HiveHookIT


Suma Shivaprasad

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message