Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 00F37200C08 for ; Thu, 12 Jan 2017 00:38:05 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id F3A65160B50; Wed, 11 Jan 2017 23:38:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 49D92160B4E for ; Thu, 12 Jan 2017 00:38:04 +0100 (CET) Received: (qmail 55766 invoked by uid 500); 11 Jan 2017 23:38:03 -0000 Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.incubator.apache.org Received: (qmail 55752 invoked by uid 99); 11 Jan 2017 23:38:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Jan 2017 23:38:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id BEF19180B1E for ; Wed, 11 Jan 2017 23:38:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.362 X-Spam-Level: X-Spam-Status: No, score=0.362 tagged_above=-999 required=6.31 tests=[RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id SAf3U8jjv4QU for ; Wed, 11 Jan 2017 23:38:01 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 990005FCA2 for ; Wed, 11 Jan 2017 23:38:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id v0BNblXl022063; Wed, 11 Jan 2017 23:37:47 GMT Message-Id: <201701112337.v0BNblXl022063@ip-10-146-233-104.ec2.internal> Date: Wed, 11 Jan 2017 23:37:47 +0000 From: "Joe McDonnell (Code Review)" To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org CC: Marcel Kornacker , Lars Volker Reply-To: joemcdonnell@cloudera.com X-Gerrit-MessageType: newpatchset Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4341=3A_Add_metadata_load_to_planner_timeline=0A?= X-Gerrit-Change-Id: I6f01a35e5f9f5007a0298acfc8e16da00ef99c6c X-Gerrit-ChangeURL: X-Gerrit-Commit: c5cab52555895b801796d6cb716b0444737b6946 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.2 archived-at: Wed, 11 Jan 2017 23:38:05 -0000 Joe McDonnell has uploaded a new patch set (#2). Change subject: IMPALA-4341: Add metadata load to planner timeline ...................................................................... IMPALA-4341: Add metadata load to planner timeline This moves the timeline from the Analyzer GlobalState to the AnalysisContext and AnalysisContext.AnalysisResult. When analysis needs to load metadata about missing tables, it marks an event noting the start of metadata load. Then, when metadata load completes (or times out), it marks an event noting that metadata load completed (or timed out). Keeping the timeline on the AnalysisContext means that it persists across attempts at analysis. AnalysisContext.AnalysisResult has a reference to the timeline, so that it persists past analyzeStmt and can be used for the rest of the planning. Here is an example output of the planner timeline after this change: Planner Timeline: 4s371ms - Metadata load started: 41.388ms (41.388ms) - Metadata load finished: 4s260ms (4s219ms) - Analysis finished: 4s296ms (35.693ms) - Equivalence classes computed: 4s315ms (19.062ms) - Single node plan created: 4s323ms (7.812ms) - Runtime filters computed: 4s323ms (777.010us) - Distributed plan created: 4s325ms (1.464ms) - Planning finished: 4s371ms (46.697ms) When there is no need to load metadata, the timeline looks like: Planner Timeline: 13.695ms - Analysis finished: 2.411ms (2.411ms) - Equivalence classes computed: 2.653ms (241.733us) - Single node plan created: 5.641ms (2.987ms) - Runtime filters computed: 5.726ms (85.204us) - Distributed plan created: 6.548ms (821.722us) - Planning finished: 13.695ms (7.147ms) Change-Id: I6f01a35e5f9f5007a0298acfc8e16da00ef99c6c --- M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/service/Frontend.java 5 files changed, 23 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/5685/2 -- To view, visit http://gerrit.cloudera.org:8080/5685 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6f01a35e5f9f5007a0298acfc8e16da00ef99c6c Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Marcel Kornacker