Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D0C2C200C5C for ; Thu, 20 Apr 2017 23:52:44 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id CF69C160B9F; Thu, 20 Apr 2017 21:52:44 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 237E4160B90 for ; Thu, 20 Apr 2017 23:52:43 +0200 (CEST) Received: (qmail 8917 invoked by uid 500); 20 Apr 2017 21:52:43 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 8887 invoked by uid 99); 20 Apr 2017 21:52:42 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Apr 2017 21:52:42 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 6D624C18AA; Thu, 20 Apr 2017 21:52:42 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.5 X-Spam-Level: *** X-Spam-Status: No, score=3.5 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, KAM_NUMSUBJECT=0.5, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id ITHsk4onAOCF; Thu, 20 Apr 2017 21:52:41 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 746035F39D; Thu, 20 Apr 2017 21:52:40 +0000 (UTC) Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id DBDACE05A2; Thu, 20 Apr 2017 21:52:39 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id CDF55C404BB; Thu, 20 Apr 2017 21:52:39 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============7807106755477196217==" MIME-Version: 1.0 Subject: Re: Review Request 58585: HIVE-16441 From: Jason Dere To: Jason Dere , j.prasanth.j@gmail.com Cc: Deepak Jaiswal , hive Date: Thu, 20 Apr 2017 21:52:39 -0000 Message-ID: <20170420215239.24123.90477@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Jason Dere X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/58585/ X-Sender: Jason Dere References: <20170420213936.24124.79376@reviews-vm2.apache.org> In-Reply-To: <20170420213936.24124.79376@reviews-vm2.apache.org> Reply-To: Jason Dere X-ReviewRequest-Repository: hive-git archived-at: Thu, 20 Apr 2017 21:52:45 -0000 --===============7807106755477196217== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit > On April 20, 2017, 9:39 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java > > Line 829 (original) > > > > > > Just a nit, but why was removeSemiJoin removed? Seems like we are now duplicating the logic to remove the semijoin. Deepak has pointed out that in the maxEntries/no stats case, all of the semijoins for that particular column expression should be removed, whereas if check fails due to TEZ_BIGTABLE_MIN_SIZE_SEMIJOIN_REDUCTION, only the particular semijoin going to that destination should be removed. - Jason ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58585/#review172548 ----------------------------------------------------------- On April 20, 2017, 6:20 p.m., Deepak Jaiswal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58585/ > ----------------------------------------------------------- > > (Updated April 20, 2017, 6:20 p.m.) > > > Review request for hive, Jason Dere and Prasanth_J. > > > Bugs: HIVE-16441 > https://issues.apache.org/jira/browse/HIVE-16441 > > > Repository: hive-git > > > Description > ------- > > De-duplicate semijoin branches in n-way joins > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java eb3eba536c > ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 9a69f90483 > ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 96525b4b66 > ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 26eda0461f > ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out a47ce6e583 > > > Diff: https://reviews.apache.org/r/58585/diff/1/ > > > Testing > ------- > > > Thanks, > > Deepak Jaiswal > > --===============7807106755477196217==--