hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-24165) CBO: Query fails after multiple count distinct rewrite
Date Fri, 23 Oct 2020 06:43:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-24165?focusedWorklogId=504048&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504048
]

ASF GitHub Bot logged work on HIVE-24165:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 23/Oct/20 06:42
            Start Date: 23/Oct/20 06:42
    Worklog Time Spent: 10m 
      Work Description: loudongfeng closed pull request #1597:
URL: https://github.com/apache/hive/pull/1597


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 504048)
    Time Spent: 20m  (was: 10m)

> CBO: Query fails after multiple count distinct rewrite 
> -------------------------------------------------------
>
>                 Key: HIVE-24165
>                 URL: https://issues.apache.org/jira/browse/HIVE-24165
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>    Affects Versions: 4.0.0
>            Reporter: Nemon Lou
>            Assignee: Nemon Lou
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-24165.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> One way to reproduce:
>  
> {code:sql}
>  CREATE TABLE test(
>  `device_id` string, 
>  `level` string, 
>  `site_id` string, 
>  `user_id` string, 
>  `first_date` string, 
>  `last_date` string,
>  `dt` string) ;
>  set hive.execution.engine=tez;
>  set hive.optimize.distinct.rewrite=true;
>  set hive.cli.print.header=true;
>  select 
>  dt,
>  site_id,
>  count(DISTINCT t1.device_id) as device_tol_cnt,
>  count(DISTINCT case when t1.first_date='2020-09-15' then t1.device_id else null end)
as device_add_cnt 
>  from test t1 where dt='2020-09-15' 
>  group by
>  dt,
>  site_id
>  ;
> {code}
>  
> Error log:  
> {code:java}
> Exception in thread "main" java.lang.AssertionError: Cannot add expression of different
type to set:
> set type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
$f2, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f3,
BIGINT $f2_0, BIGINT $f3_0) NOT NULL
> expression type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
$f2, BIGINT $f3, BIGINT $f2_0, BIGINT $f3_0) NOT NULL
> set is rel#85:HiveAggregate.HIVE.[](input=HepRelVertex#84,group={2, 3},agg#0=count($0),agg#1=count($1))
> expression is HiveProject#95
> 	at org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:411)
> 	at org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
> 	at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
> 	at org.apache.calcite.rel.rules.AggregateProjectPullUpConstantsRule.onMatch(AggregateProjectPullUpConstantsRule.java:186)
> 	at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
> 	at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556)
> 	at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:415)
> 	at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:280)
> 	at org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74)
> 	at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:211)
> 	at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:198)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:2273)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:2002)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1709)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1609)
> 	at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> 	at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1052)
> 	at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154)
> 	at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1414)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1430)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:450)
> 	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12164)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:330)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:659)
> 	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1826)
> 	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1773)
> 	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1768)
> 	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> 	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> 	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> 	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message