Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 21E3B200C11 for ; Sat, 4 Feb 2017 19:13:03 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 20585160B63; Sat, 4 Feb 2017 18:13:03 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 42F34160B54 for ; Sat, 4 Feb 2017 19:13:02 +0100 (CET) Received: (qmail 55990 invoked by uid 500); 4 Feb 2017 18:13:01 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 55981 invoked by uid 99); 4 Feb 2017 18:13:01 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 04 Feb 2017 18:13:01 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 02818C087B for ; Sat, 4 Feb 2017 18:13:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id YEfl1obbr85m for ; Sat, 4 Feb 2017 18:13:00 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id C1E4A5FC06 for ; Sat, 4 Feb 2017 18:12:59 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B433DE0146 for ; Sat, 4 Feb 2017 18:12:53 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 63B6E25290 for ; Sat, 4 Feb 2017 18:12:52 +0000 (UTC) Date: Sat, 4 Feb 2017 18:12:52 +0000 (UTC) From: "Pengcheng Xiong (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-15388) HiveParser spends lots of time in parsing queries with lots of "(" MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Sat, 04 Feb 2017 18:13:03 -0000 [ https://issues.apache.org/jira/browse/HIVE-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15852883#comment-15852883 ] Pengcheng Xiong commented on HIVE-15388: ---------------------------------------- [~hagleitn] and [~ashutoshc], i think there is some misunderstanding here. Parenthesis are now mandatory for expressions in predicate in that single q test. It is because of the "=" where "in" has higher precedence than "=". This is not saying that for every q test, we need to use parenthesis for expressions in predicate. Out of 10K+ q tests, I only discovered that single q test which needs modification and it is illegal in postgres/oracle. I also tried "select 1+1 in (1,2,3,4)" and "select (1+1) in (1,2,3,4)" in Hive. Both of them work well with my patch. Thanks. > HiveParser spends lots of time in parsing queries with lots of "(" > ------------------------------------------------------------------ > > Key: HIVE-15388 > URL: https://issues.apache.org/jira/browse/HIVE-15388 > Project: Hive > Issue Type: Improvement > Affects Versions: 2.2.0 > Reporter: Rajesh Balamohan > Assignee: Pengcheng Xiong > Attachments: HIVE-15388.01.patch, HIVE-15388.02.patch, HIVE-15388.03.patch, HIVE-15388.04.patch, HIVE-15388.05.patch, hive-15388.stacktrace.txt > > > Branch: apache-master (applicable with previous releases as well) > Queries generated via tools can have lots of "(" for "AND/OR" conditions. This causes huge delays in parsing phase when the number of expressions are high. > e.g > {noformat} > SELECT `iata`, > `airport`, > `city`, > `state`, > `country`, > `lat`, > `lon` > FROM airports > WHERE ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((`airports`.`airport` = "Thigpen" > OR `airports`.`airport` = "Astoria Regional") > OR `airports`.`airport` = "Warsaw Municipal") > OR `airports`.`airport` = "John F Kennedy Memorial") > OR `airports`.`airport` = "Hall-Miller Municipal") > OR `airports`.`airport` = "Atqasuk") > OR `airports`.`airport` = "William B Hartsfield-Atlanta Intl") > OR `airports`.`airport` = "Artesia Municipal") > OR `airports`.`airport` = "Outagamie County Regional") > OR `airports`.`airport` = "Watertown Municipal") > OR `airports`.`airport` = "Augusta State") > OR `airports`.`airport` = "Aurora Municipal") > OR `airports`.`airport` = "Alakanuk") > OR `airports`.`airport` = "Austin Municipal") > OR `airports`.`airport` = "Auburn Municipal") > OR `airports`.`airport` = "Auburn-Opelik") > OR `airports`.`airport` = "Austin-Bergstrom International") > OR `airports`.`airport` = "Wausau Municipal") > OR `airports`.`airport` = "Mecklenburg-Brunswick Regional") > OR `airports`.`airport` = "Alva Regional") > OR `airports`.`airport` = "Asheville Regional") > OR `airports`.`airport` = "Avon Park Municipal") > OR `airports`.`airport` = "Wilkes-Barre/Scranton Intl") > OR `airports`.`airport` = "Marana Northwest Regional") > OR `airports`.`airport` = "Catalina") > OR `airports`.`airport` = "Washington Municipal") > OR `airports`.`airport` = "Wainwright") > OR `airports`.`airport` = "West Memphis Municipal") > OR `airports`.`airport` = "Arlington Municipal") > OR `airports`.`airport` = "Algona Municipal") > OR `airports`.`airport` = "Chandler") > OR `airports`.`airport` = "Altus Municipal") > OR `airports`.`airport` = "Neil Armstrong") > OR `airports`.`airport` = "Angel Fire") > OR `airports`.`airport` = "Waycross-Ware County") > OR `airports`.`airport` = "Colorado City Municipal") > OR `airports`.`airport` = "Hazelhurst") > OR `airports`.`airport` = "Kalamazoo County") > OR `airports`.`airport` = "Granville") > OR `airports`.`airport` = "Silver Springs") > OR `airports`.`airport` = "Whitford") > OR `airports`.`airport` = "Biddeford Municipal") > OR `airports`.`airport` = "Sugarloaf Regional") > OR `airports`.`airport` = "Barnes Municipal") > OR `airports`.`airport` = "Columbus Municipal") > OR `airports`.`airport` = "Battle Mountain") > OR `airports`.`airport` = "Huron County Memorial") > OR `airports`.`airport` = "New Braunfels Municipal") > OR `airports`.`airport` = "Benson Municipal") > OR `airports`.`airport` = "Curtis") > OR `airports`.`airport` = "Marlboro County") > OR `airports`.`airport` = "Broken Bow Municipal") > OR `airports`.`airport` = "Virginia Tech") > OR `airports`.`airport` = "Bryce Canyon") > OR `airports`.`airport` = "Black River Falls Area") > OR `airports`.`airport` = "Boca Raton") > OR `airports`.`airport` = "Birchwood") > OR `airports`.`airport` = "Baudette International") > OR `airports`.`airport` = "Blanding Muni") > OR `airports`.`airport` = "Bradley International") > OR `airports`.`airport` = "Morrilton Municipal") > OR `airports`.`airport` = "Igor I Sikorsky Memorial") > OR `airports`.`airport` = "Broadus") > OR `airports`.`airport` = "Laurence G Hanscom") > OR `airports`.`airport` = "Southwest Michigan Regional") > OR `airports`.`airport` = "Bethel") > OR `airports`.`airport` = "Bradford Reg") > OR `airports`.`airport` = "Scotts Bluff County") > OR `airports`.`airport` = "Boeing Field/King County Intl") > OR `airports`.`airport` = "Buffalo Municipal") > OR `airports`.`airport` = "Meadows") > OR `airports`.`airport` = "Mobile Downtown") > OR `airports`.`airport` = "Virgil I Grissom Municipal") > OR `airports`.`airport` = "Silver Bay Municipal"); > {noformat} > This query would take more than minutes to parse and compile. > Temp Workaround: If all "("s in "where" condition are removed manually, this query would complete in 2-3 seconds. > Problem exists in older versions as well (but in older versions this can make HS2 unresponsive for large amount of time, as Driver ends up taking global compiler lock in this phase). -- This message was sent by Atlassian JIRA (v6.3.15#6346)