From issues-return-196625-archive-asf-public=cust-asf.ponee.io@hive.apache.org Mon Jul 27 07:17:06 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mailroute1-lw-us.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 031C3180665 for ; Mon, 27 Jul 2020 09:17:06 +0200 (CEST) Received: from mail.apache.org (localhost [127.0.0.1]) by mailroute1-lw-us.apache.org (ASF Mail Server at mailroute1-lw-us.apache.org) with SMTP id 8A5551262A4 for ; Mon, 27 Jul 2020 07:17:03 +0000 (UTC) Received: (qmail 71750 invoked by uid 500); 27 Jul 2020 07:17:02 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 71639 invoked by uid 99); 27 Jul 2020 07:17:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Jul 2020 07:17:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id A020C42FEE for ; Mon, 27 Jul 2020 07:17:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 5F52B780910 for ; Mon, 27 Jul 2020 07:17:00 +0000 (UTC) Date: Mon, 27 Jul 2020 07:17:00 +0000 (UTC) From: "ASF GitHub Bot (Jira)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Work logged] (HIVE-23893) Extract deterministic conditions for pdd when the predicate contains non-deterministic function MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-23893?focusedWorklogId=3D= 463524&page=3Dcom.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpa= nel#worklog-463524 ] ASF GitHub Bot logged work on HIVE-23893: ----------------------------------------- Author: ASF GitHub Bot Created on: 27/Jul/20 07:16 Start Date: 27/Jul/20 07:16 Worklog Time Spent: 10m=20 Work Description: letsflyinthesky opened a new pull request #1322: URL: https://github.com/apache/hive/pull/1322 =E2=80=A6ll be push down and nondeterministic filter which keeps current= position =20 ## NOTICE =20 Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HIVE-XXXXX: Fix a typo in YYY= ) For more details, please see https://cwiki.apache.org/confluence/display= /Hive/HowToContribute =20 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 463524) Time Spent: 20m (was: 10m) > Extract deterministic conditions for pdd when the predicate contains non-= deterministic function > -------------------------------------------------------------------------= ---------------------- > > Key: HIVE-23893 > URL: https://issues.apache.org/jira/browse/HIVE-23893 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer > Reporter: Zhihua Deng > Assignee: zhishui > Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Taken the following query for example, assume=C2=A0unix_timestamp is non-= deterministic before version 1.3.0: > =C2=A0 > {{SELECT}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 from_unixtime(unix_timestamp(a.first_dt), = 'yyyyMMdd') AS ft,}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 b.game_id AS game_id,}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 b.game_name AS game_name,}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 count(DISTINCT a.sha1_imei) uv}} > {{FROM}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 gamesdk_userprofile a}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 JOIN game_info_all b ON a.appid =3D b.dev_= app_id}} > {{WHERE}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 a.date =3D 20200704}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 AND from_unixtime(unix_timestamp(a.first_d= t), 'yyyyMMdd') =3D 20200704}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 AND b.date =3D 20200704}} > {{GROUP BY}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 from_unixtime(unix_timestamp(a.first_dt), = 'yyyyMMdd'),}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 b.game_id,}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 b.game_name}} > {{ORDER BY}} > {{=C2=A0 =C2=A0 =C2=A0 =C2=A0 uv DESC}} > {{LIMIT 200;}} > =C2=A0 > The predicates(a.date =3D 20200704,=C2=A0b.date =3D 20200704) are unable= to push down to join op, make the optimizer unable to prune partitions, wh= ich may=C2=A0result=C2=A0 to a full scan on tables=C2=A0gamesdk_userprofile= and=C2=A0game_info_all. -- This message was sent by Atlassian Jira (v8.3.4#803005)