Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4C8F1200B0F for ; Fri, 17 Jun 2016 22:35:19 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 4B123160A62; Fri, 17 Jun 2016 20:35:19 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 44B81160A61 for ; Fri, 17 Jun 2016 22:35:18 +0200 (CEST) Received: (qmail 32149 invoked by uid 500); 17 Jun 2016 20:35:17 -0000 Mailing-List: contact commits-help@quickstep.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@quickstep.incubator.apache.org Delivered-To: mailing list commits@quickstep.incubator.apache.org Received: (qmail 32140 invoked by uid 99); 17 Jun 2016 20:35:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jun 2016 20:35:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 153581A052D for ; Fri, 17 Jun 2016 20:35:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id ToV-UOIVYqvx for ; Fri, 17 Jun 2016 20:35:14 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id CD4BD5F39A for ; Fri, 17 Jun 2016 20:35:13 +0000 (UTC) Received: (qmail 32024 invoked by uid 99); 17 Jun 2016 20:35:13 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jun 2016 20:35:13 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 1E53AE094C; Fri, 17 Jun 2016 20:35:13 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: shixuan@apache.org To: commits@quickstep.incubator.apache.org Date: Fri, 17 Jun 2016 20:35:15 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [3/7] incubator-quickstep git commit: QUICKSTEP-20: Added parser support for SQL window aggregation function archived-at: Fri, 17 Jun 2016 20:35:19 -0000 http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/00ca1e4b/parser/preprocessed/SqlParser_gen.hpp ---------------------------------------------------------------------- diff --git a/parser/preprocessed/SqlParser_gen.hpp b/parser/preprocessed/SqlParser_gen.hpp index 71e4332..fea31d6 100644 --- a/parser/preprocessed/SqlParser_gen.hpp +++ b/parser/preprocessed/SqlParser_gen.hpp @@ -85,88 +85,96 @@ extern int quickstep_yydebug; TOKEN_CONSTRAINT = 295, TOKEN_COPY = 296, TOKEN_CREATE = 297, - TOKEN_DATE = 298, - TOKEN_DATETIME = 299, - TOKEN_DAY = 300, - TOKEN_DECIMAL = 301, - TOKEN_DEFAULT = 302, - TOKEN_DELETE = 303, - TOKEN_DELIMITER = 304, - TOKEN_DESC = 305, - TOKEN_DISTINCT = 306, - TOKEN_DOUBLE = 307, - TOKEN_DROP = 308, - TOKEN_ELSE = 309, - TOKEN_END = 310, - TOKEN_ESCAPE_STRINGS = 311, - TOKEN_EXISTS = 312, - TOKEN_EXTRACT = 313, - TOKEN_FALSE = 314, - TOKEN_FIRST = 315, - TOKEN_FLOAT = 316, - TOKEN_FOR = 317, - TOKEN_FOREIGN = 318, - TOKEN_FROM = 319, - TOKEN_FULL = 320, - TOKEN_GROUP = 321, - TOKEN_HASH = 322, - TOKEN_HAVING = 323, - TOKEN_HOUR = 324, - TOKEN_IN = 325, - TOKEN_INDEX = 326, - TOKEN_INNER = 327, - TOKEN_INSERT = 328, - TOKEN_INTEGER = 329, - TOKEN_INTERVAL = 330, - TOKEN_INTO = 331, - TOKEN_JOIN = 332, - TOKEN_KEY = 333, - TOKEN_LAST = 334, - TOKEN_LEFT = 335, - TOKEN_LIMIT = 336, - TOKEN_LONG = 337, - TOKEN_MINUTE = 338, - TOKEN_MONTH = 339, - TOKEN_NULL = 340, - TOKEN_NULLS = 341, - TOKEN_OFF = 342, - TOKEN_ON = 343, - TOKEN_ORDER = 344, - TOKEN_OUTER = 345, - TOKEN_PARTITION = 346, - TOKEN_PARTITIONS = 347, - TOKEN_PERCENT = 348, - TOKEN_PRIMARY = 349, - TOKEN_QUIT = 350, - TOKEN_RANGE = 351, - TOKEN_REAL = 352, - TOKEN_REFERENCES = 353, - TOKEN_RIGHT = 354, - TOKEN_ROW_DELIMITER = 355, - TOKEN_SECOND = 356, - TOKEN_SELECT = 357, - TOKEN_SET = 358, - TOKEN_SMA = 359, - TOKEN_SMALLINT = 360, - TOKEN_SUBSTRING = 361, - TOKEN_TABLE = 362, - TOKEN_THEN = 363, - TOKEN_TIME = 364, - TOKEN_TIMESTAMP = 365, - TOKEN_TRUE = 366, - TOKEN_TUPLESAMPLE = 367, - TOKEN_UNIQUE = 368, - TOKEN_UPDATE = 369, - TOKEN_USING = 370, - TOKEN_VALUES = 371, - TOKEN_VARCHAR = 372, - TOKEN_WHEN = 373, - TOKEN_WHERE = 374, - TOKEN_WITH = 375, - TOKEN_YEAR = 376, - TOKEN_YEARMONTH = 377, - TOKEN_EOF = 378, - TOKEN_LEX_ERROR = 379 + TOKEN_CURRENT = 298, + TOKEN_DATE = 299, + TOKEN_DATETIME = 300, + TOKEN_DAY = 301, + TOKEN_DECIMAL = 302, + TOKEN_DEFAULT = 303, + TOKEN_DELETE = 304, + TOKEN_DELIMITER = 305, + TOKEN_DESC = 306, + TOKEN_DISTINCT = 307, + TOKEN_DOUBLE = 308, + TOKEN_DROP = 309, + TOKEN_ELSE = 310, + TOKEN_END = 311, + TOKEN_ESCAPE_STRINGS = 312, + TOKEN_EXISTS = 313, + TOKEN_EXTRACT = 314, + TOKEN_FALSE = 315, + TOKEN_FIRST = 316, + TOKEN_FLOAT = 317, + TOKEN_FOLLOWING = 318, + TOKEN_FOR = 319, + TOKEN_FOREIGN = 320, + TOKEN_FROM = 321, + TOKEN_FULL = 322, + TOKEN_GROUP = 323, + TOKEN_HASH = 324, + TOKEN_HAVING = 325, + TOKEN_HOUR = 326, + TOKEN_IN = 327, + TOKEN_INDEX = 328, + TOKEN_INNER = 329, + TOKEN_INSERT = 330, + TOKEN_INTEGER = 331, + TOKEN_INTERVAL = 332, + TOKEN_INTO = 333, + TOKEN_JOIN = 334, + TOKEN_KEY = 335, + TOKEN_LAST = 336, + TOKEN_LEFT = 337, + TOKEN_LIMIT = 338, + TOKEN_LONG = 339, + TOKEN_MINUTE = 340, + TOKEN_MONTH = 341, + TOKEN_NULL = 342, + TOKEN_NULLS = 343, + TOKEN_OFF = 344, + TOKEN_ON = 345, + TOKEN_ORDER = 346, + TOKEN_OUTER = 347, + TOKEN_OVER = 348, + TOKEN_PARTITION = 349, + TOKEN_PARTITIONS = 350, + TOKEN_PERCENT = 351, + TOKEN_PRECEDING = 352, + TOKEN_PRIMARY = 353, + TOKEN_QUIT = 354, + TOKEN_RANGE = 355, + TOKEN_REAL = 356, + TOKEN_REFERENCES = 357, + TOKEN_RIGHT = 358, + TOKEN_ROW = 359, + TOKEN_ROW_DELIMITER = 360, + TOKEN_ROWS = 361, + TOKEN_SECOND = 362, + TOKEN_SELECT = 363, + TOKEN_SET = 364, + TOKEN_SMA = 365, + TOKEN_SMALLINT = 366, + TOKEN_SUBSTRING = 367, + TOKEN_TABLE = 368, + TOKEN_THEN = 369, + TOKEN_TIME = 370, + TOKEN_TIMESTAMP = 371, + TOKEN_TRUE = 372, + TOKEN_TUPLESAMPLE = 373, + TOKEN_UNBOUNDED = 374, + TOKEN_UNIQUE = 375, + TOKEN_UPDATE = 376, + TOKEN_USING = 377, + TOKEN_VALUES = 378, + TOKEN_VARCHAR = 379, + TOKEN_WHEN = 380, + TOKEN_WHERE = 381, + TOKEN_WINDOW = 382, + TOKEN_WITH = 383, + TOKEN_YEAR = 384, + TOKEN_YEARMONTH = 385, + TOKEN_EOF = 386, + TOKEN_LEX_ERROR = 387 }; #endif @@ -175,7 +183,7 @@ extern int quickstep_yydebug; union YYSTYPE { -#line 118 "../SqlParser.ypp" /* yacc.c:1915 */ +#line 119 "../SqlParser.ypp" /* yacc.c:1909 */ quickstep::ParseString *string_value_; @@ -261,13 +269,19 @@ union YYSTYPE quickstep::ParseSample *opt_sample_clause_; + quickstep::PtrList *opt_window_clause_; + quickstep::ParseWindow *window_definition_; + quickstep::PtrList *window_partition_by_list_; + quickstep::PtrList *window_order_by_list_; + quickstep::ParseFrameInfo *window_frame_info_; + quickstep::PtrList *order_commalist_; quickstep::ParseOrderByItem *order_item_; quickstep::PtrVector *with_list_; quickstep::ParseSubqueryTableReference *with_list_element_; -#line 271 "SqlParser_gen.hpp" /* yacc.c:1915 */ +#line 285 "SqlParser_gen.hpp" /* yacc.c:1909 */ }; typedef union YYSTYPE YYSTYPE; http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/00ca1e4b/parser/tests/Select.test ---------------------------------------------------------------------- diff --git a/parser/tests/Select.test b/parser/tests/Select.test index e70ee5c..79da7ee 100644 --- a/parser/tests/Select.test +++ b/parser/tests/Select.test @@ -1,5 +1,7 @@ # Copyright 2011-2015 Quickstep Technologies LLC. # Copyright 2015 Pivotal Software, Inc. +# Copyright 2016, Quickstep Research Group, Computer Sciences Department, +# University of Wisconsin—Madison. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -1709,3 +1711,121 @@ SelectStatement | +-TableReference[table=bar] +-from_clause= +-TableReference[table=test] +== + +SELECT avg(attr1) OVER w FROM test +WINDOW w AS +(PARTITION BY attr2, attr3 + ORDER BY attr4 + ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) +-- +SelectStatement ++-select_query=Select + +-select_clause=SelectList + | +-SelectListItem + | +-FunctionCall[name=avg,window_name=w] + | +-AttributeReference[attribute_name=attr1] + +-from_clause= + | +-TableReference[table=test] + +-window_list= + +-window[window_name=w] + +-frame_info=FrameInfo[frame_mode=row,num_preceding=3,num_following=3] + +-partition_by= + | +-AttributeReference[attribute_name=attr2] + | +-AttributeReference[attribute_name=attr3] + +-order_by= + +-OrderByItem[is_asc=true,nulls_first=false] + +-AttributeReference[attribute_name=attr4] +== + +SELECT avg(attr1) OVER w FROM test +WINDOW w AS +(PARTITION BY attr2, attr3 + ORDER BY attr4 DESC NULLS FIRST, attr5 ASC NULLS LAST + RANGE BETWEEN 3 PRECEDING AND CURRENT ROW) +-- +SelectStatement ++-select_query=Select + +-select_clause=SelectList + | +-SelectListItem + | +-FunctionCall[name=avg,window_name=w] + | +-AttributeReference[attribute_name=attr1] + +-from_clause= + | +-TableReference[table=test] + +-window_list= + +-window[window_name=w] + +-frame_info=FrameInfo[frame_mode=range,num_preceding=3,num_following=0] + +-partition_by= + | +-AttributeReference[attribute_name=attr2] + | +-AttributeReference[attribute_name=attr3] + +-order_by= + +-OrderByItem[is_asc=false,nulls_first=true] + | +-AttributeReference[attribute_name=attr4] + +-OrderByItem[is_asc=true,nulls_first=false] + +-AttributeReference[attribute_name=attr5] +== + +SELECT avg(attr1) OVER w1 AS avg1, sum(attr5) OVER w2 AS sum5 FROM test +WINDOW w1 AS +(PARTITION BY attr2, attr3 + ORDER BY attr4 + ROWS BETWEEN 3 PRECEDING AND 3 FOLLOWING) +WINDOW w2 AS +(PARTITION BY attr1 + ORDER BY attr6 + RANGE BETWEEN 2 PRECEDING AND CURRENT ROW) +-- +SelectStatement ++-select_query=Select + +-select_clause=SelectList + | +-SelectListItem[alias=avg1] + | | +-FunctionCall[name=avg,window_name=w1] + | | +-AttributeReference[attribute_name=attr1] + | +-SelectListItem[alias=sum5] + | +-FunctionCall[name=sum,window_name=w2] + | +-AttributeReference[attribute_name=attr5] + +-from_clause= + | +-TableReference[table=test] + +-window_list= + +-window[window_name=w1] + | +-frame_info=FrameInfo[frame_mode=row,num_preceding=3,num_following=3] + | +-partition_by= + | | +-AttributeReference[attribute_name=attr2] + | | +-AttributeReference[attribute_name=attr3] + | +-order_by= + | +-OrderByItem[is_asc=true,nulls_first=false] + | +-AttributeReference[attribute_name=attr4] + +-window[window_name=w2] + +-frame_info=FrameInfo[frame_mode=range,num_preceding=2,num_following=0] + +-partition_by= + | +-AttributeReference[attribute_name=attr1] + +-order_by= + +-OrderByItem[is_asc=true,nulls_first=false] + +-AttributeReference[attribute_name=attr6] +== + +SELECT avg(attr1) OVER +(PARTITION BY attr2, attr3 + ORDER BY attr4 DESC NULLS FIRST, attr5 ASC NULLS LAST + RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) +FROM test +-- +SelectStatement ++-select_query=Select + +-select_clause=SelectList + | +-SelectListItem + | +-FunctionCall[name=avg] + | +-AttributeReference[attribute_name=attr1] + | +-window=window + | +-frame_info=FrameInfo[frame_mode=range,num_preceding=-1, + | | num_following=0] + | +-partition_by= + | | +-AttributeReference[attribute_name=attr2] + | | +-AttributeReference[attribute_name=attr3] + | +-order_by= + | +-OrderByItem[is_asc=false,nulls_first=true] + | | +-AttributeReference[attribute_name=attr4] + | +-OrderByItem[is_asc=true,nulls_first=false] + | +-AttributeReference[attribute_name=attr5] + +-from_clause= + +-TableReference[table=test] http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/00ca1e4b/query_optimizer/resolver/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/query_optimizer/resolver/CMakeLists.txt b/query_optimizer/resolver/CMakeLists.txt index 5959879..dc7eac0 100644 --- a/query_optimizer/resolver/CMakeLists.txt +++ b/query_optimizer/resolver/CMakeLists.txt @@ -62,6 +62,7 @@ target_link_libraries(quickstep_queryoptimizer_resolver_Resolver quickstep_parser_ParseSubqueryExpression quickstep_parser_ParseSubqueryTableReference quickstep_parser_ParseTableReference + quickstep_parser_ParseWindow quickstep_queryoptimizer_OptimizerContext quickstep_queryoptimizer_Validator quickstep_queryoptimizer_expressions_AggregateFunction http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/00ca1e4b/query_optimizer/resolver/Resolver.cpp ---------------------------------------------------------------------- diff --git a/query_optimizer/resolver/Resolver.cpp b/query_optimizer/resolver/Resolver.cpp index 78985a0..ffc173a 100644 --- a/query_optimizer/resolver/Resolver.cpp +++ b/query_optimizer/resolver/Resolver.cpp @@ -58,6 +58,7 @@ #include "parser/ParseSubqueryExpression.hpp" #include "parser/ParseSubqueryTableReference.hpp" #include "parser/ParseTableReference.hpp" +#include "parser/ParseWindow.hpp" #include "query_optimizer/OptimizerContext.hpp" #include "query_optimizer/Validator.hpp" #include "query_optimizer/expressions/AggregateFunction.hpp" @@ -2366,6 +2367,12 @@ E::ScalarPtr Resolver::resolveFunctionCall( ExpressionResolutionInfo *expression_resolution_info) { std::string function_name = ToLower(parse_function_call.name()->value()); + // TODO(Shixuan): Add support for window aggregation function. + if (parse_function_call.isWindow()) { + THROW_SQL_ERROR_AT(&parse_function_call) + << "Window Aggregation Function is not supported currently"; + } + // First check for the special case COUNT(*). bool count_star = false; if (parse_function_call.star() != nullptr) {