atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-274) No lineage is recorded for creating a table using LIKE
Date Fri, 20 Nov 2015 09:13:10 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015466#comment-15015466
] 

Hemanth Yamijala commented on ATLAS-274:
----------------------------------------

I am able to replicate this behavior. However, thinking through more, a LIKE is a different
clause than, say a CTAS. LIKE doesn't copy the data of the parent table, it only copies the
table definition and attributes.

Quote from the [Hive manual|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTable]:

bq. The LIKE form of CREATE TABLE allows you to copy an existing table definition exactly
(without copying its data).

Given this, and my understanding that lineage captures how *data* is transformed from one
form to another, I don't think it is appropriate to show the relationship between the two
tables linked by lineage.

If we are interested in capturing the fact that one table's definition is derived from the
other, there is value in that, but it should be expressed as another relationship in Atlas
possibly. Don't know what that relationship should be. Can we discuss and take this forward?

> No lineage is recorded for creating a table using LIKE
> ------------------------------------------------------
>
>                 Key: ATLAS-274
>                 URL: https://issues.apache.org/jira/browse/ATLAS-274
>             Project: Atlas
>          Issue Type: Bug
>    Affects Versions: 0.6-incubating
>            Reporter: Ayub Khan
>            Assignee: Hemanth Yamijala
>         Attachments: application.log
>
>
> Seems like no lineage is recorded for creating a table using LIKE
> example:
> create table table_1242 LIKE table_1240
> Input graph query
> {noformat}
> curl 'http://localhost:21000/api/atlas/lineage/hive/table/primary.default.table_1242/inputs/graph'
-H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80
Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer: http://localhost:21000/'
-H 'Cookie: JSESSIONID=15b7c1abmgzpsv2gax1e68sor' -H 'Connection: keep-alive' --compressed
| python -m json.tool
> {
>     "requestId": "qtp983814036-14184 - f97fb1cb-be0b-4eaa-8338-5c1543803438",
>     "results": {
>         "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>         "typeName": "__tempQueryResultStruct267",
>         "values": {
>             "edges": {},
>             "vertices": {}
>         }
>     },
>     "tableName": "primary.default.table_1242"
> }
> {noformat}
> Output graph query
> {noformat}
> curl 'http://localhost:21000/api/atlas/lineage/hive/table/primary.default.table_1242/outputs/graph'
-H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80
Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer: http://localhost:21000/'
-H 'Cookie: JSESSIONID=15b7c1abmgzpsv2gax1e68sor' -H 'Connection: keep-alive' --compressed
| python -m json.tool
> {
>     "requestId": "qtp983814036-14184 - 14e8cbc2-c2df-42b5-b506-51649fe2705b",
>     "results": {
>         "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>         "typeName": "__tempQueryResultStruct261",
>         "values": {
>             "edges": {},
>             "vertices": {}
>         }
>     },
>     "tableName": "primary.default.table_1242"
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message