hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navis Ryu" <navis....@nexr.com>
Subject Review Request 29390: CBO changes constant to column type
Date Wed, 24 Dec 2014 06:27:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29390/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-9195
    https://issues.apache.org/jira/browse/HIVE-9195


Repository: hive-git


Description
-------

Making testcase for HIVE-8613, I've found CBO changes constant expr to column expr. For example
(only in test mode).
{code}
CREATE TABLE bucket (key double, value string) CLUSTERED BY (key) SORTED BY (key DESC)  INTO
4 BUCKETS STORED AS TEXTFILE;
load data local inpath '../../data/files/srcsortbucket1outof4.txt' INTO TABLE bucket;
load data local inpath '../../data/files/srcsortbucket2outof4.txt' INTO TABLE bucket;
load data local inpath '../../data/files/srcsortbucket3outof4.txt' INTO TABLE bucket;
load data local inpath '../../data/files/srcsortbucket4outof4.txt' INTO TABLE bucket;

select percentile_approx(case when key < 100 then cast('NaN' as double) else key end, 0.5)
from bucket;
{code}

It works in shell but in TestCliDriver, that induces argument type exception creating udaf
evaluator, which expects constant OI for second argument.
{noformat}
2014-12-22 17:03:31,433 ERROR parse.SemanticAnalyzer (SemanticAnalyzer.java:analyzeInternal(10102))
- CBO failed, skipping CBO.
org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: The second argument must be a constant,
but double was passed instead.
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileApprox.getEvaluator(GenericUDAFPercentileApprox.java:146)
        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1160)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3794)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:4467)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapAggrNoSkew(SemanticAnalyzer.java:5536)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8884)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10086)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
        at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
        at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:877)
        at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:136)
        at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23(TestCliDriver.java:120)
{noformat}


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
99b2950 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java bd610d9 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeColumnListDesc.java f8738cd 
  ql/src/test/queries/clientpositive/udaf_percentile_approx_23.q fa1dd6e 
  ql/src/test/results/clientpositive/udaf_percentile_approx_23.q.out 32ae74b 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyVoidObjectInspector.java
d63d75a 
  serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java e844979

  serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableVoidObjectInspector.java
02b672a 

Diff: https://reviews.apache.org/r/29390/diff/


Testing
-------


Thanks,

Navis Ryu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message