atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-528) support drop table, view
Date Thu, 07 Apr 2016 06:45:25 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229797#comment-15229797
] 

Hemanth Yamijala commented on ATLAS-528:
----------------------------------------

Applied the patch on top of ATLAS-527 and reviewed it. A few comments:

* The handling of truncate table will create a single process with no inputs and a single
table as output. We should validate if this type of lineage view is OK for a end user. I agree
though that capturing the information is important.
* On the same note, is the query string captured correctly for the process. Can we enhance
the truncate test to validate this?
* If a table / view being dropped is not present in Atlas, this would generate a 404 and we
would loose the information. Maybe we can file a bug and track this for later?
* We should also test what happens if a user executes a {{drop table if exists non_existent_table}}
- do we get called?
* There are break statements missing after {{ALTERTABLE_LOCATION}} and {{DROPTABLE/DROPVIEW}}.
While the latter are the last branches of the switch, it is still safer to add IMO.


> support drop table, view
> ------------------------
>
>                 Key: ATLAS-528
>                 URL: https://issues.apache.org/jira/browse/ATLAS-528
>             Project: Atlas
>          Issue Type: Sub-task
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>             Fix For: 0.7-incubating
>
>         Attachments: ATLAS-528.patch
>
>
> Drop table and view requires soft deletes. The reason is whenever a table is dropped
, it also may have an associated lineage which consists of a hive_process which N input tables
and an output table. If the table is dropped, the lineage edge is also dropped resulting in
incorrect lineage history. 
> With soft deletes, the expected behaviour is to changes the table status to "deleted"
and when the table is recreated through a create table statement, then create another vertex/entity
for that table with the new state. Also,  the lineage for this newly recreated table should
be a new hive_process and should not reuse the existing entity/vertex even though the hive
statement for that process is the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message