hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-12662) StackOverflowError in HiveSortJoinReduceRule when limit=0
Date Thu, 02 Jun 2016 10:18:05 GMT

     [ https://issues.apache.org/jira/browse/HIVE-12662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jesus Camacho Rodriguez updated HIVE-12662:
-------------------------------------------
    Fix Version/s:     (was: 2.1.0)

> StackOverflowError in HiveSortJoinReduceRule when limit=0
> ---------------------------------------------------------
>
>                 Key: HIVE-12662
>                 URL: https://issues.apache.org/jira/browse/HIVE-12662
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 2.1.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>             Fix For: 2.0.0
>
>         Attachments: HIVE-12662.patch
>
>
> L96 of HiveSortJoinReduceRule, you will see 
> {noformat}
>     // Finally, if we do not reduce the input size, we bail out
>     if (RexLiteral.intValue(sortLimit.fetch)
>             >= RelMetadataQuery.getRowCount(reducedInput)) {
>       return false;
>     }
> {noformat}
> It is using “ RelMetadataQuery.getRowCount” which is always at least 1. This is the
problem that we resolved in CALCITE-987.
> To confirm this, I just run the q file :
> {noformat}
> set hive.mapred.mode=nonstrict;
> set hive.optimize.limitjointranspose=true;
> set hive.optimize.limitjointranspose.reductionpercentage=1f;
> set hive.optimize.limitjointranspose.reductiontuples=0;
> explain
> select *
> from src src1 right outer join (
>   select *
>   from src src2 left outer join src src3
>   on src2.value = src3.value) src2
> on src1.key = src2.key
> limit 0;
> {noformat}
>   And I got
> {noformat}
> 2015-12-11T10:21:04,435 ERROR [c1efb099-f900-46dc-9f74-97af0944a99d main[]]: parse.CalcitePlanner
(CalcitePlanner.java:genOPTree(301)) - CBO failed, skipping CBO.
> java.lang.RuntimeException: java.lang.StackOverflowError
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.rethrowCalciteException(CalcitePlanner.java:749)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:645)
~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:264)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10076)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:223)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:456) [hive-exec-2.1.0-SNAPSHOT.jar:?]
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:310) [hive-exec-2.1.0-SNAPSHOT.jar:?]
>         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1138) [hive-exec-2.1.0-SNAPSHOT.jar:?]
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1187) [hive-exec-2.1.0-SNAPSHOT.jar:?]
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1063) [hive-exec-2.1.0-SNAPSHOT.jar:?]
> {noformat}
> via [~pxiong]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message