atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sarath Subramanian <sar...@apache.org>
Subject Re: Review Request 62457: ATLAS-2157: HiveHook error while processing temporary tables
Date Thu, 21 Sep 2017 18:18:03 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62457/#review185910
-----------------------------------------------------------


Fix it, then Ship it!





addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
Line 552 (original), 550 (patched)
<https://reviews.apache.org/r/62457/#comment262246>

    do we need this additional get on db? doesn't db have a valid value assigned from switch
statement above?


- Sarath Subramanian


On Sept. 21, 2017, 10:39 a.m., Madhan Neethiraj wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62457/
> -----------------------------------------------------------
> 
> (Updated Sept. 21, 2017, 10:39 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Sarath Subramanian, and Suma
Shivaprasad.
> 
> 
> Bugs: ATLAS-2157
>     https://issues.apache.org/jira/browse/ATLAS-2157
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> As reported in user list, executing Hive SQL that use temprorary tables (for example
insert statement) causes Atlas hook to report the following failure. The fix is to ignore
this error, as it doesn't result in loss of functionality.
> 
> 
> 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table values__tmp__table__1
not found: default.values__tmp__table__1 table not found
> 17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due to error
> java.lang.reflect.UndeclaredThrowableException
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
>         at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess() failed.
>         at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
>         at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
>         at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
>         at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>         ... 6 more
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.processHiveEntity() failed.
>         at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
>         at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
>         ... 12 more
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities()
failed.
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
>         at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
>         ... 13 more
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities()
failed.
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
>         ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
values__tmp__table__1
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
>         at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
>         ... 15 more
> 
> 
> Diffs
> -----
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 7dc2e2f1

> 
> 
> Diff: https://reviews.apache.org/r/62457/diff/1/
> 
> 
> Testing
> -------
> 
> Verified that the error is not reported in the following cases:
> - CREATE TEMPORARY TABLE tmp1(c1 string);
> - insert into tmp1 values('1');
> 
> Also verified that lineage details are recorded correctly - for both table and column
level lineages:
> - CREATE TABLE t1(c1 STRING, c2 STRING);
> - CREATE TABLE s1(c1 STRING, c2 STRING);
> - CREATE VIEW v_t1_s1 AS SELECT (t.c1 + s.c1) c1, (t.c2 + s.c2) c2 FROM t1 t, s1 s;
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message