From dev-return-13870-archive-asf-public=cust-asf.ponee.io@calcite.apache.org Tue Oct 29 02:57:59 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 41833180638 for ; Tue, 29 Oct 2019 03:57:59 +0100 (CET) Received: (qmail 92775 invoked by uid 500); 29 Oct 2019 02:57:57 -0000 Mailing-List: contact dev-help@calcite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@calcite.apache.org Delivered-To: mailing list dev@calcite.apache.org Received: (qmail 92763 invoked by uid 99); 29 Oct 2019 02:57:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2019 02:57:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 39273C117F for ; Tue, 29 Oct 2019 02:57:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id D1pMLm0RKL7J for ; Tue, 29 Oct 2019 02:57:53 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.167.176; helo=mail-oi1-f176.google.com; envelope-from=yuzhao.cyz@gmail.com; receiver= Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 391F4BC570 for ; Tue, 29 Oct 2019 02:57:53 +0000 (UTC) Received: by mail-oi1-f176.google.com with SMTP id r27so1576787oij.7 for ; Mon, 28 Oct 2019 19:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:message-id:in-reply-to:references:subject:mime-version; bh=hzMvG8kA5r/RZSblT2eR9tyGot+wl9Cla7UXDJfQrcc=; b=r/295WHy1D4TfDXzhqG0DOvByCXY2jwAijomm9EKO1ksWYU3ruD5/uivkV3AYhVKZ7 PWCPJJqBFXQ0F9HRHOlhj/U2hFP/Wl/NVOdY70eOVFbCa+0AqiuwLYP4aobsgQ7GhTCA GT1vd+ajiMEcszCm5J7Vb/fNCcAvrkBam3vq+w8WoqTXRzM5RiS94P//Mbyklklp/Wwh fyc33wlIEWGLzxS7eiqr/qFhY+/r94ckhppIBTqNlo0Xc18a1Ej7mRCxQgc1VCbha4tX CP57xst5bxRdX9EJDRsSHgR9mvwE7tv5NW7SDeO1fg3QSxV+g6K9+yQlnr3fAY2D4knt DhQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version; bh=hzMvG8kA5r/RZSblT2eR9tyGot+wl9Cla7UXDJfQrcc=; b=OthhWLGtWNg09t4ANViCaEmP+p5h44VYcL8uzjOcmu2Gs8WmmUwbfiszaDZ7MFY71z EaA4hvF63M5Su3ub+ib2M5yHJzsLTzp15E7Bq6sgiuePgQvLwT4UBHGc1JkkgFTVhp6w 11fEOpscbfAz+xdcGIMhPPH2q6qUCLD1oZNKzyHQa0dMKaZhaR16NCZCpz5YpXLQA8cO gXy2wj8S5hymUQXhpBh893DGs8KFfTYqytMTuaX5/hBgz+SORu7r8MODyiA5BOHfJRky 8SBMc8MVBkVSVtx8w00UOPintPHXDPm2eORcgf9hKFDyQYEUp01T90rw1/xkVGU4cTjS bdZQ== X-Gm-Message-State: APjAAAVPLnah3ul31b0Gi2OFFVqQ6dgEGjw+oA/MUjZODX7DU1yMNRmg HKAQgDZ6oWizEbC6BB6eRwGJl7wz X-Google-Smtp-Source: APXvYqyptev0MFj84h7E8rCQVAi5qLh2AZnA6WnhnqItBJYyMgni+GGSqKr8Qe/A4sdks0daX60m2Q== X-Received: by 2002:aca:520a:: with SMTP id g10mr2116780oib.1.1572317872431; Mon, 28 Oct 2019 19:57:52 -0700 (PDT) Received: from [30.5.24.252] ([205.204.117.30]) by smtp.gmail.com with ESMTPSA id u1sm1587718oie.37.2019.10.28.19.57.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Oct 2019 19:57:52 -0700 (PDT) Date: Tue, 29 Oct 2019 10:57:41 +0800 From: Danny Chan To: dev@calcite.apache.org Message-ID: In-Reply-To: <8F1ACA3E-86D4-45AF-97FA-BF5CCF941C62@gmail.com> References: <8F1ACA3E-86D4-45AF-97FA-BF5CCF941C62@gmail.com> Subject: Re: Same rules fired for logical and physical nodes X-Readdle-Message-ID: ba01e37b-45f5-4ab2-970c-2bc939bbd1d1@Spark MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5db7aaac_1ca0c5fa_3dd" --5db7aaac_1ca0c5fa_3dd Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Thanks Xiening Dai for bringing up this topic =7E I agree that for most of the planning rules, only matching logical nodes = are enough. There are 2 concerts from my side: =E2=80=A2 We have some plan rewrite that indeed happens on the =E2=80=9Cp= hysical=E2=80=9D node, so some the rules may be still useful for such cas= e, i.e. the ProjectMergeRule =E2=80=A2 I=E2=80=99m a little worried about the compatibility, maybe we = can rule the test cases like Apache Druid which give us more confidence t= hat this is the right way to go Best, Danny Chan =E5=9C=A8 2019=E5=B9=B410=E6=9C=8829=E6=97=A5 +0800 AM12:48=EF=BC=8CXieni= ng Dai =EF=BC=8C=E5=86=99=E9=81=93=EF=BC=9A > Hi all, > > While I was looking at CALCITE-2970, I noticed that some of the rules a= re fired for both logical and physical nodes. =46or example, ProjectMerge= Rule matches Project.class, so it=E2=80=99s fired for LogicalProject. But= then after LogicalProject is converted into EnummerableProject, the same= rule is fired again for the physical rels. Same for EnumerableLimitRule,= SortRemoveConstantKeysRule, etc. > > This seems to be unnecessary. When ProjectMerge is applied to LogicalPr= oject nodes, we already generate all possible alternatives with merged pr= ojects. We just need to convert the LogicalProject into EnumerableProject= . There=E2=80=99s no need to merge EnumerableProject again. > > If I update those rules to only match logical nodes, the planning time = of the case in CALCITE-2970 is reduced =7E30%. > > Any thoughts=3F --5db7aaac_1ca0c5fa_3dd--