Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A2EAB200D2F for ; Wed, 1 Nov 2017 19:23:20 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id A165E160BEA; Wed, 1 Nov 2017 18:23:20 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C04BF1609EC for ; Wed, 1 Nov 2017 19:23:19 +0100 (CET) Received: (qmail 84299 invoked by uid 500); 1 Nov 2017 18:23:18 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 84288 invoked by uid 99); 1 Nov 2017 18:23:18 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Nov 2017 18:23:18 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id A148F1A0C6D; Wed, 1 Nov 2017 18:23:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3 X-Spam-Level: *** X-Spam-Status: No, score=3 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id sxay48aI4nYC; Wed, 1 Nov 2017 18:23:15 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 5AB0A5FDAB; Wed, 1 Nov 2017 18:23:15 +0000 (UTC) Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 05990E05B7; Wed, 1 Nov 2017 18:23:15 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id 191F1C40096; Wed, 1 Nov 2017 18:23:14 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============8172669635888973988==" MIME-Version: 1.0 Subject: Review Request 63470: HIVE-17767 Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN From: Vineet Garg To: Ashutosh Chauhan Cc: hive , Vineet Garg Date: Wed, 01 Nov 2017 18:23:14 -0000 Message-ID: <20171101182314.32367.76255@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Vineet Garg X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/63470/ X-Sender: Vineet Garg X-ReviewBoard-Diff-For: ql/src/test/results/clientpositive/vector_mapjoin_reduce.q.out Reply-To: Vineet Garg X-ReviewRequest-Repository: hive-git archived-at: Wed, 01 Nov 2017 18:23:20 -0000 --===============8172669635888973988== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63470/ ----------------------------------------------------------- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-17767 https://issues.apache.org/jira/browse/HIVE-17767 Repository: hive-git Description ------- This patch does the following: * Added back earlier patch to generate value generator * Added logic to rewrite EXISTS/IN correlated subqueries into LEFT SEMI JOIN * Remove SemiJoinTransposeRule (This rule pushes semi join underneath its left join which might not be semantically correct thing to do) Diffs ----- itests/src/test/resources/testconfiguration.properties 462f332e99 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSubQRemoveRelBuilder.java 3a1897f4aa ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java 62125f0fb7 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java 2dca6a25ac ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 80351bef87 ql/src/test/queries/clientpositive/subquery_exists.q 19c42f0c29 ql/src/test/queries/clientpositive/subquery_in.q 4ba170a706 ql/src/test/results/clientpositive/constprog_partitioner.q.out 87618df902 ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out 87e08fbcde ql/src/test/results/clientpositive/llap/explainuser_1.q.out 6e55acf0d8 ql/src/test/results/clientpositive/llap/lineage3.q.out 66cc6ad5a0 ql/src/test/results/clientpositive/llap/subquery_exists.q.out e206f0851e ql/src/test/results/clientpositive/llap/subquery_in.q.out af42131bc2 ql/src/test/results/clientpositive/llap/subquery_multi.q.out 96fe17a05a ql/src/test/results/clientpositive/llap/subquery_notin.q.out 8e2ca937af ql/src/test/results/clientpositive/llap/subquery_scalar.q.out c89d053b4a ql/src/test/results/clientpositive/llap/subquery_select.q.out 118f6ebccf ql/src/test/results/clientpositive/llap/subquery_views.q.out a9a81133b5 ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out 4e6f00f6b7 ql/src/test/results/clientpositive/masking_12.q.out 540c53e825 ql/src/test/results/clientpositive/masking_3.q.out 1114c80676 ql/src/test/results/clientpositive/masking_4.q.out 527da21610 ql/src/test/results/clientpositive/perf/spark/query10.q.out eb3a2f6699 ql/src/test/results/clientpositive/perf/spark/query16.q.out b74d721d41 ql/src/test/results/clientpositive/perf/spark/query35.q.out 8759b71b8c ql/src/test/results/clientpositive/perf/spark/query69.q.out e4430beaac ql/src/test/results/clientpositive/perf/spark/query94.q.out 43b8c77bdc ql/src/test/results/clientpositive/perf/tez/query10.q.out cf3651b35b ql/src/test/results/clientpositive/perf/tez/query14.q.out b2a45f155a ql/src/test/results/clientpositive/perf/tez/query16.q.out a7b710d6e1 ql/src/test/results/clientpositive/perf/tez/query23.q.out 7112de61d9 ql/src/test/results/clientpositive/perf/tez/query35.q.out a72f57816e ql/src/test/results/clientpositive/perf/tez/query69.q.out 591f3fcdb0 ql/src/test/results/clientpositive/perf/tez/query94.q.out 7674aa7f7c ql/src/test/results/clientpositive/semijoin5.q.out 533c077f58 ql/src/test/results/clientpositive/spark/constprog_partitioner.q.out b89f9f5905 ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out 76c74d9ab7 ql/src/test/results/clientpositive/spark/subquery_exists.q.out dafe5b6d5b ql/src/test/results/clientpositive/spark/subquery_in.q.out 471c2ccd94 ql/src/test/results/clientpositive/spark/subquery_multi.q.out ff519fda09 ql/src/test/results/clientpositive/spark/subquery_notin.q.out 1b2c0880ae ql/src/test/results/clientpositive/spark/subquery_scalar.q.out de005ada82 ql/src/test/results/clientpositive/spark/subquery_select.q.out 7d3a16b6ee ql/src/test/results/clientpositive/spark/subquery_views.q.out 91e39913a7 ql/src/test/results/clientpositive/spark/vector_mapjoin_reduce.q.out 81af937e97 ql/src/test/results/clientpositive/subquery_exists.q.out c9f2a79041 ql/src/test/results/clientpositive/subquery_exists_having.q.out 2c41ff6c33 ql/src/test/results/clientpositive/subquery_in_having.q.out 6893442b61 ql/src/test/results/clientpositive/subquery_notexists.q.out 329573e8e1 ql/src/test/results/clientpositive/subquery_notexists_having.q.out 4d2b2fc873 ql/src/test/results/clientpositive/subquery_notin_having.q.out c321fe69ed ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 5c306f6b47 ql/src/test/results/clientpositive/vector_mapjoin_reduce.q.out ddea584990 Diff: https://reviews.apache.org/r/63470/diff/1/ Testing ------- Thanks, Vineet Garg --===============8172669635888973988==--