Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2123218C21 for ; Fri, 15 May 2015 22:22:38 +0000 (UTC) Received: (qmail 78265 invoked by uid 500); 15 May 2015 22:22:30 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 78168 invoked by uid 500); 15 May 2015 22:22:30 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 78158 invoked by uid 99); 15 May 2015 22:22:30 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 May 2015 22:22:30 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id EB6D0C52CE for ; Fri, 15 May 2015 22:22:29 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.9 X-Spam-Level: ** X-Spam-Status: No, score=2.9 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id leLOlZKLWjA3 for ; Fri, 15 May 2015 22:22:23 +0000 (UTC) Received: from mail-la0-f53.google.com (mail-la0-f53.google.com [209.85.215.53]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id AB38B20D6D for ; Fri, 15 May 2015 22:22:22 +0000 (UTC) Received: by lagv1 with SMTP id v1so141339744lag.3 for ; Fri, 15 May 2015 15:20:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=5lt2bh30wF2fkDQPDI4PZ1yTY/UGFLyV7SbU7Oi0fl8=; b=mud0BAGf4uefAnJXehZBlAh/01E9jFfRK4dcTYxS5ZZy+o9Ha8u3/PRqGB/U26lxWT bMZuS6F5nrk9UImmc/byWA7Chw3+LsxxXyNrHYXmQheAuBYZuKGnpINzss0Bi60mBFKh Zu1Ps9tvIpQaQGg9q6S099RWtuXReiF2fgXJqmU7CuA2kqTDFfbAfFVk5LoqyIa7X1mX 8QfpJdvnfcs8nYqnH3Aq21nZXU0sTnMAbccX+Ii2JdaLm4/eMozLn5nl2NnhKxz0vQiO S0EDY/B6y1Miw2f0de1JaXMcVjV8pNFwyxGI/SYYjPHK4pFE0BD9BJ1LyFCK3429Yumh o/Tw== X-Received: by 10.152.18.227 with SMTP id z3mr8763033lad.6.1431728450649; Fri, 15 May 2015 15:20:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.30.77 with HTTP; Fri, 15 May 2015 15:20:30 -0700 (PDT) In-Reply-To: References: From: Alexander Pivovarov Date: Fri, 15 May 2015 15:20:30 -0700 Message-ID: Subject: Re: query uses WITH blocks and throws exception if run as Oozie hive action (hive-0.13.1) To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e01493976b7454c05162641c6 --089e01493976b7454c05162641c6 Content-Type: text/plain; charset=UTF-8 Looks like it was fixed in hive-0.14 https://issues.apache.org/jira/browse/HIVE-7079 On Fri, May 15, 2015 at 2:26 PM, Alexander Pivovarov wrote: > Hi Everyone > > I'm using hive-0.13.1 (HDP-2.1.5) and getting the following stacktrace > if run my query (which has WITH block) via Oozie. (BTW, the query works > fine in CLI) > > I can't put exact query but the structure is similar to > > create table my_consumer > as > with sacusaloan as (select distinct e,f,g from E) > > select A.a, A.b, A.c, > if(sacusaloan.id is null, 0, 1) as sacusaloan_status > from (select a,b,c from A) A > left join sacusaloan on (...) > > 8799 [main] INFO hive.ql.parse.ParseDriver - Parse Completed > 8799 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - > 8799 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - > 8834 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Starting Semantic Analysis > 8837 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Creating table wk_qualified_outsource_loan_consumer position=13 > 8861 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Completed phase 1 of Semantic Analysis > 8861 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 8865 [main] ERROR hive.ql.metadata.Hive - NoSuchObjectException(message:default.sacusaloan table not found) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) > at com.sun.proxy.$Proxy18.getTable(Unknown Source) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:976) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:918) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1263) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1232) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9252) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) > at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359) > at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456) > at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466) > at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) > at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:316) > at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:277) > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) > at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > 8872 [main] ERROR hive.ql.metadata.Hive - NoSuchObjectException(message:default.recallloan table not found) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) > at com.sun.proxy.$Proxy18.getTable(Unknown Source) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:976) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:918) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1263) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1232) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9252) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) > at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359) > at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456) > at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466) > at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) > at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:316) > at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:277) > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) > at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > 9700 [main] ERROR hive.ql.metadata.Hive - NoSuchObjectException(message:default.loanmob table not found) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) > at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) > at com.sun.proxy.$Proxy18.getTable(Unknown Source) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:976) > at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:918) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1263) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1232) > at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9252) > at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) > at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359) > at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456) > at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466) > at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) > at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:316) > at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:277) > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) > at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > 9708 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 9708 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 9798 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 9798 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 9798 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 9852 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 9852 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 9852 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 9876 [main] INFO org.apache.hadoop.hive.ql.exec.Utilities - Create dirs hdfs://hadev/tmp/hive-svc-yarn/hive_2015-05-15_13-58-05_500_5122268870471366216-1 with permission rwxrwxrwx recursive false > 9894 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Completed getting MetaData in Semantic Analysis > 10277 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 10289 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 10290 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 10294 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 10294 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 10294 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables > 10320 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries > 10321 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 10321 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables > 10816 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Set stats collection dir : hdfs://hadev/tmp/hive-svc-yarn/hive_2015-05-15_13-58-05_500_5122268870471366216-1/-ext-10002 > > > --089e01493976b7454c05162641c6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Looks like it was fixed in hive-0.14
https://issues.apache.org/jira/br= owse/HIVE-7079

On Fri, May 15, 2015 at 2:26 PM, Alexander Pivovarov <apiv= ovarov@gmail.com> wrote:
Hi Everyone

I'm using hive-0.13.1= =C2=A0=C2=A0 (HDP-2.1.5) and getting the following stacktrace if run my que= ry (which has WITH block) via Oozie.=C2=A0=C2=A0 (BTW, the query works fine= in CLI)

I can't put exact query but the structure is simi= lar to
create table my_consumer
as
with sacusaloan = as (select distinct e,f,g from E)

select A.a, A.b, A.c,
if(sacusaloan.id is null, 0= , 1) as sacusaloan_status
from (select a,b,c from A) A
left join sacu= saloan on (...)

8799 [main] INFO hive.ql.parse.ParseDriver - Parse= Completed 8799 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - </PERFLOG = method=3Dparse start=3D1431723485500 end=3D1431723485602 duration=3D102 fro= m=3Dorg.apache.hadoop.hive.ql.Driver> 8799 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG m= ethod=3DsemanticAnalyze from=3Dorg.apache.hadoop.hive.ql.Driver> 8834 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Start= ing Semantic Analysis 8837 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Creat= ing table wk_qualified_outsource_loan_consumer position=3D13 8861 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Compl= eted phase 1 of Semantic Analysis 8861 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for source tables 8865 [main] ERROR hive.ql.metadata.Hive - NoSuchObjectException(message:de= fault.sacusaloan table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_ge= t_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tab= le(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaS= toreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(Retryin= gMetaStoreClient.java:89) at com.sun.proxy.$Proxy18.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:976) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:918) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAn= alyzer.java:1263) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAn= alyzer.java:1232) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(Semant= icAnalyzer.java:9252) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemant= icAnalyzer.java:327) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268= ) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:316) at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:277) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:2= 25) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati= on.java:1594) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) 8872 [main] ERROR hive.ql.metadata.Hive - NoSuchObjectException(message:de= fault.recallloan table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_ge= t_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tab= le(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaS= toreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(Retryin= gMetaStoreClient.java:89) at com.sun.proxy.$Proxy18.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:976) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:918) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAn= alyzer.java:1263) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAn= alyzer.java:1232) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(Semant= icAnalyzer.java:9252) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemant= icAnalyzer.java:327) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268= ) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:316) at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:277) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:2= 25) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati= on.java:1594) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) 9700 [main] ERROR hive.ql.metadata.Hive - NoSuchObjectException(message:de= fault.loanmob table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_resu= lt.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_ge= t_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tab= le(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaS= toreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(Retryin= gMetaStoreClient.java:89) at com.sun.proxy.$Proxy18.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:976) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:918) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAn= alyzer.java:1263) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAn= alyzer.java:1232) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(Semant= icAnalyzer.java:9252) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemant= icAnalyzer.java:327) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268= ) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359) at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456) at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:316) at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:277) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:2= 25) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati= on.java:1594) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) 9708 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for subqueries 9708 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for source tables 9798 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for subqueries 9798 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for destination tables 9798 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for source tables 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for subqueries 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for destination tables 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for source tables 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for subqueries 9815 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for source tables 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for subqueries 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for destination tables 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for destination tables 9827 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for source tables 9852 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for subqueries 9852 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for destination tables 9852 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get m= etadata for destination tables 9876 [main] INFO org.apache.hadoop.hive.ql.exec.Utilities - Create dirs h= dfs://hadev/tmp/hive-svc-yarn/hive_2015-05-15_13-58-05_500_5122268870471366= 216-1 with permission rwxrwxrwx recursive false 9894 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Compl= eted getting MetaData in Semantic Analysis 10277 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for source tables 10289 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for subqueries 10290 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for destination tables 10294 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for source tables 10294 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for subqueries 10294 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for source tables 10320 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for subqueries 10321 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for destination tables 10321 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get = metadata for destination tables 10816 [main] INFO org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Set = stats collection dir : hdfs://hadev/tmp/hive-svc-yarn/hive_2015-05-15_13-58= -05_500_5122268870471366216-1/-ext-10002


--089e01493976b7454c05162641c6--