atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "aflyary (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-2893) Atlas Column Lineage of Hive Hook to support the hive old version (before hive version 1.2.1 after the patch for HIVE-13112)
Date Fri, 28 Sep 2018 02:45:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631283#comment-16631283
] 

aflyary commented on ATLAS-2893:
--------------------------------

I refer to the hive source code, when we set the altas hook class at the hive-site.xml with
the name 'org.apache.atlas.hive.hook.HiveHook', hive optimizer would not set the linegae
info into the hookContext, after the patch HIVE-13112 already fix this. But early hive version
still could not work, so i go ahead and dive into the hive lineageState of SessionState, maybe
we could get the private info of the LineageCtx to work out the lineage dependency,  then
get the column lineage.

> Atlas Column Lineage of Hive Hook to support the hive old version (before hive version
1.2.1 after the patch for HIVE-13112)
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-2893
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2893
>             Project: Atlas
>          Issue Type: Improvement
>          Components: atlas-intg
>    Affects Versions: 1.0.0
>            Reporter: aflyary
>            Priority: Major
>             Fix For: 1.1.0
>
>
>         Right now atlas hook for apache hive of column lineage is limited, only "Column
level lineage works with Hive version 1.2.1 after the patch for HIVE-13112 is applied to
Hive source" could support the hive column lineage.
>         In some product env, hive version still very old, but we should also support
the hive column lineage.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message