Return-Path: 
 <dev-return-63862-apmail-ambari-dev-archive=ambari.apache.org@ambari.apache.org>
X-Original-To: apmail-ambari-dev-archive@www.apache.org
Delivered-To: apmail-ambari-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 8C05A1737D
	for <apmail-ambari-dev-archive@www.apache.org>;
 Fri, 20 Mar 2015 16:57:38 +0000 (UTC)
Received: (qmail 21172 invoked by uid 500); 20 Mar 2015 16:57:38 -0000
Delivered-To: apmail-ambari-dev-archive@ambari.apache.org
Received: (qmail 21130 invoked by uid 500); 20 Mar 2015 16:57:38 -0000
Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@ambari.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@ambari.apache.org>
List-Post: <mailto:dev@ambari.apache.org>
List-Id: <dev.ambari.apache.org>
Reply-To: dev@ambari.apache.org
Delivered-To: mailing list dev@ambari.apache.org
Received: (qmail 21111 invoked by uid 99); 20 Mar 2015 16:57:38 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Mar 2015 16:57:38 +0000
Date: Fri, 20 Mar 2015 16:57:38 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: dev@ambari.apache.org
Message-ID: <JIRA.12783674.1426869850000.158663.1426870658340@Atlassian.JIRA>
In-Reply-To: <JIRA.12783674.1426869850000@Atlassian.JIRA>
References: <JIRA.12783674.1426869850000@Atlassian.JIRA>
 <JIRA.12783674.1426869850860@arcas>
Subject: [jira] [Commented] (AMBARI-10160) Hive Metastore alert check "show
 databases" results in a Tez session causing RM to be unresponsive
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/AMBARI-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371616#comment-14371616 ] 

Hadoop QA commented on AMBARI-10160:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12705971/AMBARI-10160.patch
  against trunk revision .

    {color:red}-1 patch{color}.  Top-level trunk compilation may be broken.

Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/2104//console

This message is automatically generated.

> Hive Metastore alert check "show databases" results in a Tez session causing RM to be unresponsive
> --------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-10160
>                 URL: https://issues.apache.org/jira/browse/AMBARI-10160
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.0.0
>            Reporter: Jonathan Hurley
>            Assignee: Jonathan Hurley
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: AMBARI-10160.patch
>
>
> The HS2 check does this:
> {code}
> export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf hive.metastore.uris=thrift://revo3.hortonworks.local:9083 -e 'show databases;'
> {code}
> Looks like this does this to hive:
> {code}
> [3/19/15, 7:47:37 PM] Paul Codding: 2015-03-19 18:44:22,959 INFO  [main]: parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: show databases
> 2015-03-19 18:44:23,801 INFO  [main]: parse.ParseDriver (ParseDriver.java:parse(206)) - Parse Completed
> 2015-03-19 18:44:23,809 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=parse start=1426808662932 end=1426808663809 duration=877 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:23,818 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,002 INFO  [main]: ql.Driver (Driver.java:compile(427)) - Semantic Analysis Completed
> 2015-03-19 18:44:24,003 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=semanticAnalyze start=1426808663818 end=1426808664003 duration=185 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,386 INFO  [main]: exec.ListSinkOperator (Operator.java:initialize(346)) - Initializing Self OP[0]
> 2015-03-19 18:44:24,394 INFO  [main]: exec.ListSinkOperator (Operator.java:initializeChildren(419)) - Operator 0 OP initialized
> 2015-03-19 18:44:24,395 INFO  [main]: exec.ListSinkOperator (Operator.java:initialize(394)) - Initialization Done 0 OP
> 2015-03-19 18:44:24,419 INFO  [main]: ql.Driver (Driver.java:getSchema(235)) - Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
> 2015-03-19 18:44:24,420 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=compile start=1426808662818 end=1426808664420 duration=1602 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,420 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,420 INFO  [main]: ql.Driver (Driver.java:execute(1285)) - Starting command: show databases
> 2015-03-19 18:44:25,090 INFO  [main]: impl.TimelineClientImpl (TimelineClientImpl.java:serviceInit(285)) - Timeline service address: http://revo2.hortonworks.local:8188/ws/v1/timeline/
> 2015-03-19 18:44:25,093 INFO  [main]: hooks.ATSHook (ATSHook.java:<init>(87)) - Created ATS Hook
> 2015-03-19 18:44:25,093 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,098 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1426808665093 end=1426808665098 duration=5 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,103 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=TimeToSubmit start=1426808662818 end=1426808665103 duration=2285 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,103 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,103 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,114 INFO  [main]: ql.Driver (Driver.java:launchTask(1602)) - Starting task [Stage-0:DDL] in serial mode
> 2015-03-19 18:44:25,309 INFO  [main]: exec.DDLTask (DDLTask.java:showDatabases(2173)) - results : 1
> 2015-03-19 18:44:25,317 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=runTasks start=1426808665103 end=1426808665317 duration=214 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,319 INFO  [main]: hooks.ATSHook (ATSHook.java:<init>(87)) - Created ATS Hook
> 2015-03-19 18:44:25,319 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,320 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1426808665319 end=1426808665320 duration=1 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,321 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.execute start=1426808664420 end=1426808665321 duration=901 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,330 INFO  [main]: ql.Driver (SessionState.java:printInfo(824)) - OK
> 2015-03-19 18:44:25,331 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,331 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks start=1426808665331 end=1426808665331 duration=0 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,332 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.run start=1426808662817 end=1426808665332 duration=2515 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,353 INFO  [main]: Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 2015-03-19 18:44:25,536 INFO  [main]: mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
> 2015-03-19 18:44:25,635 INFO  [main]: exec.ListSinkOperator (Operator.java:close(595)) - 0 finished. closing... 
> 2015-03-19 18:44:25,635 INFO  [main]: exec.ListSinkOperator (Operator.java:close(613)) - 0 Close done
> 2015-03-19 18:44:25,660 INFO  [main]: CliDriver (SessionState.java:printInfo(824)) - Time taken: 2.529 seconds, Fetched: 1 row(s)
> 2015-03-19 18:44:25,660 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,662 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks start=1426808665660 end=1426808665662 duration=2 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,708 INFO  [main]: tez.TezSessionPoolManager (TezSessionPoolManager.java:close(174)) - Closing tez session default? false
> 2015-03-19 18:44:25,709 INFO  [main]: tez.TezSessionState (TezSessionState.java:close(263)) - Closing Tez Session
> 2015-03-19 18:44:25,710 INFO  [main]: client.TezClient (TezClient.java:stop(434)) - Shutting down Tez Session, sessionName=HIVE-f96da6a9-ad75-4949-b8ba-a9e3b65278d3, applicationId=application_1426808089446_0009
> [3/19/15, 7:47:50 PM] Paul Codding: salient
> [3/19/15, 7:47:50 PM] Paul Codding: 2015-03-19 18:44:22,959 INFO  [main]: parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: show databases
> 2015-03-19 18:44:23,801 INFO  [main]: parse.ParseDriver (ParseDriver.java:parse(206)) - Parse Completed
> {code}
> Changing this check to use the MR enginer instead of TEZ corrects the problem:
> {code}
> export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf hive.metastore.uris=thrift://revo3.hortonworks.local:9083 --hiveconf hive.execution.engine=mr -e 'show databases;'
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)