Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D79A4200BF9 for ; Sun, 8 Jan 2017 08:01:56 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id D61F8160B36; Sun, 8 Jan 2017 07:01:56 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D040E160B2C for ; Sun, 8 Jan 2017 08:01:55 +0100 (CET) Received: (qmail 74387 invoked by uid 500); 8 Jan 2017 07:01:55 -0000 Mailing-List: contact notifications-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list notifications@asterixdb.apache.org Received: (qmail 74378 invoked by uid 99); 8 Jan 2017 07:01:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 Jan 2017 07:01:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 836901A0699 for ; Sun, 8 Jan 2017 07:01:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.919 X-Spam-Level: X-Spam-Status: No, score=0.919 tagged_above=-999 required=6.31 tests=[SPF_FAIL=0.919] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id aR0uswS04qGI for ; Sun, 8 Jan 2017 07:01:51 +0000 (UTC) Received: from unhygienix.ics.uci.edu (unhygienix.ics.uci.edu [128.195.14.130]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 6D0505F1EE for ; Sun, 8 Jan 2017 07:01:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by unhygienix.ics.uci.edu (Postfix) with ESMTP id BD98E2420EB; Sat, 7 Jan 2017 23:01:49 -0800 (PST) Date: Sat, 7 Jan 2017 23:01:47 -0800 From: "Wenhai Li (Code Review)" To: Michael Blow , Preston Carman , Jenkins CC: Yingyi Bu , Yingyi Bu , Jianfeng Jia Reply-To: lwhaymail@yahoo.com X-Gerrit-MessageType: newpatchset Subject: Change in asterixdb[master]: RangeGenerator aggfunc for the numeric/asciiString datatype ... X-Gerrit-Change-Id: I450d0962fbeacfb2b6ab9fae0750f025ef17ba01 X-Gerrit-ChangeURL: X-Gerrit-Commit: b46aa46e0c351ba4bc270679764f567e4a7d1443 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.8.4 Message-Id: <20170108070149.BD98E2420EB@unhygienix.ics.uci.edu> archived-at: Sun, 08 Jan 2017 07:01:57 -0000 Hello Michael Blow, Preston Carman, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/806 to look at the new patch set (#53). Change subject: RangeGenerator aggfunc for the numeric/asciiString datatype based on parallel streaming histogram. ...................................................................... RangeGenerator aggfunc for the numeric/asciiString datatype based on parallel streaming histogram. The following commits from your working branch will be included: Extract and review for the streaming histogram based parallel sort. 1. Add the streaming histogram for hyracks datatype. 2. Include the ASCII histogram. 3. Inline the rangegenerator inside the AggFuncs. 4. Add the testcases including the random, zipfan, gaussian int/short/long/double/string. 5. Support the null omittion. Change-Id: I450d0962fbeacfb2b6ab9fae0750f025ef17ba01 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAggFuncIntoStandaloneAggregateRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java A asterixdb/asterix-app/data/skew/zipfan1.tbl A asterixdb/asterix-app/data/skew/zipfan2.tbl A asterixdb/asterix-app/data/skew/zipfanmissing_1.tbl A asterixdb/asterix-app/data/skew/zipfanmissing_2.tbl A asterixdb/asterix-app/data/skew/zipfannull_1.tbl A asterixdb/asterix-app/data/skew/zipfannull_2.tbl A asterixdb/asterix-app/data/tpch0.001/lineitem-part1.tbl A asterixdb/asterix-app/data/tpch0.001/lineitem-part2.tbl M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_float/avg_float.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int16/avg_int16.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int32/avg_int32.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int64/avg_int64.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/avg_int8/avg_int8.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/count_01/count_01.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-avg_01/global-avg_01.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/global-rg/global-rg.1.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_double/local-avg_double.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_float/local-avg_float.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.1.ddl.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int16/local-avg_int16.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int32/local-avg_int32.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int64/local-avg_int64.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-avg_int8/local-avg_int8.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/local-rg/local-rg.1.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double-null/rg-double-null.1.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg-double/rg-double.1.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double/rg_double.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double_missing/rg_double_missing.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double_missing/rg_double_missing.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_double_missing/rg_double_missing.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16/rg_int16.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_missing/rg_int16_missing.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_missing/rg_int16_missing.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_missing/rg_int16_missing.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_null/rg_int16_null.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_null/rg_int16_null.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_null/rg_int16_null.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_seminull/rg_int16_seminull.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_seminull/rg_int16_seminull.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int16_seminull/rg_int16_seminull.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32/rg_int32.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32_missing/rg_int32_missing.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32_missing/rg_int32_missing.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int32_missing/rg_int32_missing.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64/rg_int64.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64_missing/rg_int64_missing.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64_missing/rg_int64_missing.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_int64_missing/rg_int64_missing.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_shipdate/rg_shipdate.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_shipdate/rg_shipdate.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_shipdate/rg_shipdate.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string/rg_string.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string_missing/rg_string_missing.1.ddl.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string_missing/rg_string_missing.2.update.aql A asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/rg_string_missing/rg_string_missing.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_double/sum_double.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_float/sum_float.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int16/sum_int16.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int64/sum_int64.3.query.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries/aggregate/sum_int8/sum_int8.3.query.aql A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/global-rg/global-avg.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/local-rg/local-rg.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg-double-null/rg-double-null.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg-double/rg-double.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_double/rg_double.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_double_missing/rg_double_missing.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16/rg_int16.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16_missing/rg_int16_missing.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16_null/rg_int16_null.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int16_seminull/rg_int16_seminull.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int32/rg_int32.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int32_missing/rg_int32_missing.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int64/rg_int64.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_int64_missing/rg_int64_missing.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_shipdate/rg_shipdate.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_string/rg_string.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/rg_string_missing/rg_string_missing.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/GlobalRangeGeneratorTypeComputer.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/LocalRangeGeneratorTypeComputer.java A asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RangeGeneratorTypeComputer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/ScalarVersionOfAggregateResultType.java M asterixdb/asterix-om/src/test/java/org/apache/asterix/om/typecomputer/ExceptionTest.java M asterixdb/asterix-om/src/test/java/org/apache/asterix/om/typecomputer/TypeComputerTest.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/histogram/AsterixStreamingHistogram.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarRangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractRangeGuiderAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalRangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalRangeGuiderAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalRangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/LocalRangeGuiderAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/RangeGuiderAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/RangeGuiderAggregateFunction.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceCombinerRule.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/HistogramType.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IHeapList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IHistogram.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/IQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/AbstractQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/AbstractStreamingHistogram.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/DominantQuantile.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/GenericQuantile.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/GenericQuantileComparator.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/MinDifferenceQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/structures/MinSumCountQuantileList.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/range/util/HistogramUtils.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/range/TypeTranslateTest.java A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/prefix.tbl A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/zipfan1.tbl A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/data/skew/zipfan2.tbl M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/pom.xml A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HistogramAlgorithmTests.java A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HistogramFieldType.java A hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/HyracksStreamingHistogram.java 127 files changed, 22,546 insertions(+), 24 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/06/806/53 -- To view, visit https://asterix-gerrit.ics.uci.edu/806 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I450d0962fbeacfb2b6ab9fae0750f025ef17ba01 Gerrit-PatchSet: 53 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Wenhai Li Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Jianfeng Jia Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Preston Carman Gerrit-Reviewer: Wenhai Li Gerrit-Reviewer: Yingyi Bu Gerrit-Reviewer: Yingyi Bu