Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7FC5B177BE for ; Sat, 8 Nov 2014 07:59:34 +0000 (UTC) Received: (qmail 94568 invoked by uid 500); 8 Nov 2014 07:59:34 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 94498 invoked by uid 500); 8 Nov 2014 07:59:34 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 94481 invoked by uid 500); 8 Nov 2014 07:59:34 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 94476 invoked by uid 99); 8 Nov 2014 07:59:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 08 Nov 2014 07:59:34 +0000 Date: Sat, 8 Nov 2014 07:59:33 +0000 (UTC) From: "Thejas M Nair (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-8785: -------------------------------- Attachment: HIVE-8785.5.patch Yup, race condition! Serializing now - HIVE-8785.5.patch - includes Gunther's 1 line change to TezTask logging > HiveServer2 LogDivertAppender should be more selective for beeline getLogs > -------------------------------------------------------------------------- > > Key: HIVE-8785 > URL: https://issues.apache.org/jira/browse/HIVE-8785 > Project: Hive > Issue Type: Bug > Reporter: Gopal V > Assignee: Thejas M Nair > Fix For: 0.14.0 > > Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch, HIVE-8785.4.patch, HIVE-8785.4.patch, HIVE-8785.5.patch > > > A simple query run via beeline JDBC like {{explain select count(1) from testing.foo;}} produces 50 lines of output which looks like > {code} > 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo; > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select count(1) from testing.foo > 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic Analysis > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination tables > 14/11/06 00:35:59 INFO ql.Context: New scratch dir is hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1 > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in Semantic Analysis > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002 > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16) > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15) > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14) > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13) > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12) > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11) > 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10) > 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 oldColExprMap: {VALUE._col0=Column[_col0]} > 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 newColExprMap: {VALUE._col0=Column[_col0]} > 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation > 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > +----------------------------+--+ > | Explain | > +----------------------------+--+ > | STAGE DEPENDENCIES: | > | Stage-0 is a root stage | > | | > | STAGE PLANS: | > | Stage: Stage-0 | > | Fetch Operator | > | limit: 1 | > | Processor Tree: | > | ListSink | > | | > +----------------------------+--+ > 10 rows selected (0.1 seconds) > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) from testing.foo > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO ql.Driver: Starting task [Stage-1:EXPLAIN] in serial mode > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO ql.Driver: OK > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > 14/11/06 00:35:59 INFO log.PerfLogger: > {code} > A more complex query like Query27 produces 800+ lines of unnecessary logging. > This is unreadable and in-fact slows down the beeline JDBC client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)