From issues-return-4251-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Mon Jan 28 12:37:05 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 77D0118067E for ; Mon, 28 Jan 2019 12:37:04 +0100 (CET) Received: (qmail 59956 invoked by uid 500); 28 Jan 2019 11:37:03 -0000 Mailing-List: contact issues-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list issues@phoenix.apache.org Received: (qmail 59947 invoked by uid 99); 28 Jan 2019 11:37:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Jan 2019 11:37:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 3775A180947 for ; Mon, 28 Jan 2019 11:37:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 1HV2kR6OgCQk for ; Mon, 28 Jan 2019 11:37:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 1F39360E12 for ; Mon, 28 Jan 2019 11:37:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 90E79E00CA for ; Mon, 28 Jan 2019 11:37:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2FD5B2437E for ; Mon, 28 Jan 2019 11:37:00 +0000 (UTC) Date: Mon, 28 Jan 2019 11:37:00 +0000 (UTC) From: "Hadoop QA (JIRA)" To: issues@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PHOENIX-4009) Run UPDATE STATISTICS command by using MR integration on snapshots MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753919#comment-16753919 ] Hadoop QA commented on PHOENIX-4009: ------------------------------------ {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12956540/PHOENIX-4009.master.003.patch against master branch at commit 573ec9f73a86d236a6e69a2bac23f177a29f3c44. ATTACHMENT ID: 12956540 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 release audit{color}. The applied patch generated 1 release audit warnings (more than the master's current 0 warnings). {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + protected BaseStatsCollectorIT(boolean userTableNamespaceMapped, boolean collectStatsOnSnapshot) { + protected BaseStatsCollectorIT(boolean mutable, String transactionProvider, boolean columnEncoded) { + boolean userTableNamespaceMapped, boolean columnEncoded, boolean collectStatsOnSnapshot) { + updateStatisticsSql += " SET \"" + QueryServices.STATS_GUIDEPOST_WIDTH_BYTES_ATTRIB + "\" = " + guidePostWidth; + String guidePostWidth, String localPhysicalTableName) throws Exception { + LOG.info("Successfully created snapshot " + snapshotName + " for " + localPhysicalTableName); + conn.createStatement().execute("ALTER TABLE " + fullTableName + " SET GUIDE_POSTS_WIDTH = " + guidePostWidth); + String q = "SELECT SUM(GUIDE_POSTS_ROW_COUNT) FROM SYSTEM.STATS WHERE PHYSICAL_NAME = '" + physicalTableName + "'"; + conn.createStatement().executeUpdate("DELETE FROM SYSTEM.STATS WHERE PHYSICAL_NAME = '" + physicalTableName + "'"); + q = "SELECT SUM(GUIDE_POSTS_ROW_COUNT) FROM SYSTEM.STATS WHERE PHYSICAL_NAME = '" + physicalTableName + "'"; {color:red}-1 core tests{color}. The patch failed these unit tests: ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NullIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConcurrentMutationsIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UngroupedIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.Array1IT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SpillableGroupByIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExplainPlanWithStatsDisabledIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DistinctPrefixFilterIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitReverseScanIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.join.HashJoinMoreIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DateTimeIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryWithOffsetIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AggregateIT Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2274//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2274//artifact/patchprocess/patchReleaseAuditWarnings.txt Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2274//console This message is automatically generated. > Run UPDATE STATISTICS command by using MR integration on snapshots > ------------------------------------------------------------------ > > Key: PHOENIX-4009 > URL: https://issues.apache.org/jira/browse/PHOENIX-4009 > Project: Phoenix > Issue Type: Bug > Reporter: Samarth Jain > Assignee: Karan Mehta > Priority: Major > Labels: stats, statsCollection > Attachments: PHOENIX-4009.4.x-HBase-1.4.001.patch, PHOENIX-4009.4.x-HBase-1.4.002.patch, PHOENIX-4009.4.x-HBase-1.4.003.addendum.patch, PHOENIX-4009.master.001.patch, PHOENIX-4009.master.002.patch, PHOENIX-4009.master.003.patch > > Time Spent: 13h 20m > Remaining Estimate: 0h > > Now that we have the capability to run queries against table snapshots through our map reduce integration, we can utilize this capability for stats collection too. This would make our stats collection more resilient, resource aware and less resource intensive. The bulk of the plumbing is already in place. We would need to make sure that the integration doesn't barf when the query is an UPDATE STATISTICS command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)