atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vimal Sharma (JIRA)" <>
Subject [jira] [Commented] (ATLAS-642) import-hive should create the lineage for external tables
Date Wed, 08 Jun 2016 12:43:21 GMT


Vimal Sharma commented on ATLAS-642:

We can create the lineage by extracting metadata from Hive table. We need to create a hive
process with HDFS path and table as input and output respectively.

I have submit a patch with this change. We can revisit the task of getting the "SHOW CREATE
TABLE" query for a table in another JIRA.

> import-hive should create the lineage for external tables
> ---------------------------------------------------------
>                 Key: ATLAS-642
>                 URL:
>             Project: Atlas
>          Issue Type: Sub-task
>            Reporter: Suma Shivaprasad
>            Assignee: Vimal Sharma
>             Fix For: 0.7-incubating
>         Attachments: ATLAS-642.patch
> import-hive can also start showing lineage between HDFS paths and tables. 
> However the create table process should also register the exact query that was  used
to create the table. Show create table does that by reconstructing the query. Need to figure
out if we can potentially use that to create the query

This message was sent by Atlassian JIRA

View raw message