Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DD4B517218 for ; Wed, 29 Oct 2014 17:09:21 +0000 (UTC) Received: (qmail 42811 invoked by uid 500); 29 Oct 2014 17:09:21 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 42739 invoked by uid 500); 29 Oct 2014 17:09:21 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 42720 invoked by uid 99); 29 Oct 2014 17:09:21 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Oct 2014 17:09:21 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 7EAC21DF900; Wed, 29 Oct 2014 17:09:25 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============8996386004477218476==" MIME-Version: 1.0 Subject: Re: Review Request 27283: Compile time skew join optimization doesn't work with auto map join From: "Xuefu Zhang" To: "Szehon Ho" , "Xuefu Zhang" Cc: "hive" , "Rui Li" Date: Wed, 29 Oct 2014 17:09:25 -0000 Message-ID: <20141029170925.7143.19557@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Xuefu Zhang" X-ReviewGroup: hive X-ReviewRequest-URL: https://reviews.apache.org/r/27283/ X-Sender: "Xuefu Zhang" References: <20141029025949.7137.24711@reviews.apache.org> In-Reply-To: <20141029025949.7137.24711@reviews.apache.org> Reply-To: "Xuefu Zhang" X-ReviewRequest-Repository: hive-git --===============8996386004477218476== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27283/#review59001 ----------------------------------------------------------- ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java static method? Maybe we can create such clone() method in QBJoinTree class. ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java Static method? Same to below ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java The method seems to be recursive, but I'm not sure it's correct. We call the same method for the parents and children of the current operator. When a parent or child operator of the current operator is processed, wouldn't the process go back to the current operator again? Recursion should be done in one direction and should have clear end condition. - Xuefu Zhang On Oct. 29, 2014, 2:59 a.m., Rui Li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27283/ > ----------------------------------------------------------- > > (Updated Oct. 29, 2014, 2:59 a.m.) > > > Review request for hive, Szehon Ho and Xuefu Zhang. > > > Bugs: HIVE-8610 > https://issues.apache.org/jira/browse/HIVE-8610 > > > Repository: hive-git > > > Description > ------- > > This patch adds QBJoinTree and colExprMap for the cloned join operator tree in SkewJoinOptimizer, so that CommonJoinResolver can properly convert the cloned join to map join. > The added tests are copied from skewjoinopt*.q, except that auto map join is enabled. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/SkewJoinOptimizer.java e87c41b > ql/src/test/queries/clientpositive/skewjoin_mapjoin1.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin10.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin11.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin2.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin3.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin4.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin5.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin6.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin7.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin8.q PRE-CREATION > ql/src/test/queries/clientpositive/skewjoin_mapjoin9.q PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin1.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin10.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin11.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin2.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin3.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin4.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin5.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin6.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin7.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin8.q.out PRE-CREATION > ql/src/test/results/clientpositive/skewjoin_mapjoin9.q.out PRE-CREATION > > Diff: https://reviews.apache.org/r/27283/diff/ > > > Testing > ------- > > > Thanks, > > Rui Li > > --===============8996386004477218476==--