atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Madhan Neethiraj (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ATLAS-2157) Atlas Hive hook runs into unhandled exceptions from beeline if there are temp tables involved
Date Thu, 21 Sep 2017 17:42:00 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Madhan Neethiraj updated ATLAS-2157:
------------------------------------
    Attachment: ATLAS-2157.patch

Review board URL: https://reviews.apache.org/r/62457

> Atlas Hive hook runs into unhandled exceptions from beeline if there are temp tables
involved
> ---------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-2157
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2157
>             Project: Atlas
>          Issue Type: Bug
>          Components: atlas-intg
>    Affects Versions: 0.8-incubating, 0.7.1-incubating
>         Environment: HDP 2.6.2
>            Reporter: Sailaja Navvluru
>            Assignee: Madhan Neethiraj
>             Fix For: 0.9-incubating, 0.8.2
>
>         Attachments: ATLAS-2157.patch
>
>
> Opening JIRA as requested by [~madhan.neethiraj] in the mailing list
> *Repro steps:*
> Insert into a Hive table from beeline, insert goes through fine but Atlas Hive hook throws
errors.
> *Detailed description pasted from mailing list*:
> created a table from both Hive CLI and beeline and inserted a row with
> Atlas Hive hook enabled. From Hive CLI the insert goes through fine without any errors
but
> from beeline I see errors for a temp table not found although insert itself succeeds
and temp
> table shows up in table listing. Do I need to do anything extra for beeline/HS2 connections
> when Atlas is enabled ? 
>  
> hive> create table test1(c1 int, c2 string);
> OK
> Time taken: 2.108 seconds
> hive> insert into test1 values(1,'v');
> Query ID = hive_20170920092542_d8ec0a4b-b88d-4d87-b59d-790f2eae6792
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id application_1505852425742_0004)
>  
> --------------------------------------------------------------------------------
>         VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
> --------------------------------------------------------------------------------
> Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
> --------------------------------------------------------------------------------
> VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 4.68 s
> --------------------------------------------------------------------------------
> Loading data to table default.test1
> Table default.test1 stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3]
> OK
> Time taken: 6.7 seconds
> hive> select * from test1;
> OK
> 1       v
> Time taken: 0.112 seconds, Fetched: 1 row(s)
> hive>
>  
> Beeline version 1.2.1000.2.6.2.0-133 by Apache Hive
> 0: jdbc:hive2://> create table s1(c1 int,c2 string);
> OK
> No rows affected (2.345 seconds)
>  
> 0: jdbc:hive2://> insert into s1 values(1,'v');
> 17/09/20 09:28:24 [main]: ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri]
> to create a keyProvider !!
> Query ID = hive_20170920092824_14bfdec4-dfe8-4b97-b5be-0af74de055ce
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id application_1505852425742_0005)
>  
> Loading data to table default.s1
> Table default.s1 stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3]
> OK
> No rows affected (498.295 seconds)
> 0: jdbc:hive2://>
> 0: jdbc:hive2://> 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table values__tmp__table__1
> not found: default.values__tmp__table__1 table not found
> 17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due to error
> java.lang.reflect.UndeclaredThrowableException
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
>         at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess() failed.
>         at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
>         at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
>         at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
>         at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>         ... 6 more
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.processHiveEntity() failed.
>         at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
>         at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
>         ... 12 more
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities()
failed.
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
>         at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
>         ... 13 more
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities()
failed.
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
>         ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
values__tmp__table__1
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
>         ... 15 more
> 0: jdbc:hive2://> select * from s1;
> OK
> +--------+--------+--+
> | s1.c1  | s1.c2  |
> +--------+--------+--+
> | 1      | v      |
> +--------+--------+--+
> 1 row selected (0.259 seconds)
> 0: jdbc:hive2://>
> 0: jdbc:hive2://> show tables;
> OK
> +------------------------+--+
> |        tab_name        |
> +------------------------+--+
> | s1                     |
> | test1                  |
> | values__tmp__table__1  |
> +------------------------+--+
> 3 rows selected (0.075 seconds)
>   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message