hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Panagiotis Garefalakis (Jira)" <j...@apache.org>
Subject [jira] [Assigned] (HIVE-24165) CBO: Query fails after multiple count distinct rewrite
Date Tue, 15 Sep 2020 08:37:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Panagiotis Garefalakis reassigned HIVE-24165:
---------------------------------------------

    Assignee: Nemon Lou

> CBO: Query fails after multiple count distinct rewrite 
> -------------------------------------------------------
>
>                 Key: HIVE-24165
>                 URL: https://issues.apache.org/jira/browse/HIVE-24165
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>    Affects Versions: 4.0.0
>            Reporter: Nemon Lou
>            Assignee: Nemon Lou
>            Priority: Major
>         Attachments: HIVE-24165.patch
>
>
> One way to reproduce:
>  
> {code:sql}
>  CREATE TABLE test(
>  `device_id` string, 
>  `level` string, 
>  `site_id` string, 
>  `user_id` string, 
>  `first_date` string, 
>  `last_date` string,
>  `dt` string) ;
>  set hive.execution.engine=tez;
>  set hive.optimize.distinct.rewrite=true;
>  set hive.cli.print.header=true;
>  select 
>  dt,
>  site_id,
>  count(DISTINCT t1.device_id) as device_tol_cnt,
>  count(DISTINCT case when t1.first_date='2020-09-15' then t1.device_id else null end)
as device_add_cnt 
>  from test t1 where dt='2020-09-15' 
>  group by
>  dt,
>  site_id
>  ;
> {code}
>  
> Error log:  
> {code:java}
> Exception in thread "main" java.lang.AssertionError: Cannot add expression of different
type to set:
> set type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
$f2, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" $f3,
BIGINT $f2_0, BIGINT $f3_0) NOT NULL
> expression type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary"
$f2, BIGINT $f3, BIGINT $f2_0, BIGINT $f3_0) NOT NULL
> set is rel#85:HiveAggregate.HIVE.[](input=HepRelVertex#84,group={2, 3},agg#0=count($0),agg#1=count($1))
> expression is HiveProject#95
> 	at org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:411)
> 	at org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
> 	at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
> 	at org.apache.calcite.rel.rules.AggregateProjectPullUpConstantsRule.onMatch(AggregateProjectPullUpConstantsRule.java:186)
> 	at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
> 	at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556)
> 	at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:415)
> 	at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:280)
> 	at org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74)
> 	at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:211)
> 	at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:198)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:2273)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:2002)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1709)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1609)
> 	at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> 	at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1052)
> 	at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154)
> 	at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1414)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1430)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:450)
> 	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12164)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:330)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:659)
> 	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1826)
> 	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1773)
> 	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1768)
> 	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> 	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
> 	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> 	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
> 	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> 	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message