Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9F297200CF9 for ; Sun, 13 Aug 2017 02:11:10 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9DACA164A42; Sun, 13 Aug 2017 00:11:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id BE01B164A3E for ; Sun, 13 Aug 2017 02:11:09 +0200 (CEST) Received: (qmail 17553 invoked by uid 500); 13 Aug 2017 00:11:07 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 17539 invoked by uid 99); 13 Aug 2017 00:11:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Aug 2017 00:11:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 53A41C2380 for ; Sun, 13 Aug 2017 00:11:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -98.001 X-Spam-Level: X-Spam-Status: No, score=-98.001 tagged_above=-999 required=6.31 tests=[KAM_BADIPHTTP=2, NORMAL_HTTP_TO_IP=0.001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id ZiCjUspHEbzr for ; Sun, 13 Aug 2017 00:11:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id A04785F522 for ; Sun, 13 Aug 2017 00:11:05 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id DE9A1E099F for ; Sun, 13 Aug 2017 00:11:03 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0A0A42140B for ; Sun, 13 Aug 2017 00:11:01 +0000 (UTC) Date: Sun, 13 Aug 2017 00:11:01 +0000 (UTC) From: "Hive QA (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-17308) Improvement in join cardinality estimation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sun, 13 Aug 2017 00:11:10 -0000 [ https://issues.apache.org/jira/browse/HIVE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124756#comment-16124756 ] Hive QA commented on HIVE-17308: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12881632/HIVE-17308.1.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 11004 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move] (batchId=243) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only] (batchId=243) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only] (batchId=243) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_alt_syntax] (batchId=75) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_2] (batchId=57) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_4] (batchId=79) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=159) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only] (batchId=170) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=169) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=100) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query24] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query29] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query50] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query54] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query72] (batchId=235) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85] (batchId=235) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_alt_syntax] (batchId=135) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_cond_pushdown_2] (batchId=126) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_cond_pushdown_4] (batchId=136) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=222) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=180) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=180) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6373/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6373/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6373/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 29 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12881632 - PreCommit-HIVE-Build > Improvement in join cardinality estimation > ------------------------------------------ > > Key: HIVE-17308 > URL: https://issues.apache.org/jira/browse/HIVE-17308 > Project: Hive > Issue Type: Improvement > Components: Query Planning > Reporter: Vineet Garg > Assignee: Vineet Garg > Attachments: HIVE-17308.1.patch > > > Currently during logical planning join cardinality is estimated assuming no correlation among join keys (This estimation is done using exponential backoff). Physical planning on the other hand consider correlation for multi keys and uses different estimation. We should consider correlation during logical planning as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)