spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenchen Fan (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-20329) Resolution error when HAVING clause uses GROUP BY expression that involves implicit type coercion
Date Fri, 21 Apr 2017 02:08:04 GMT

     [ https://issues.apache.org/jira/browse/SPARK-20329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wenchen Fan resolved SPARK-20329.
---------------------------------
       Resolution: Fixed
         Assignee: Herman van Hovell
    Fix Version/s: 2.3.0
                   2.2.0

> Resolution error when HAVING clause uses GROUP BY expression that involves implicit type
coercion
> -------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-20329
>                 URL: https://issues.apache.org/jira/browse/SPARK-20329
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.0
>            Reporter: Josh Rosen
>            Assignee: Herman van Hovell
>            Priority: Blocker
>             Fix For: 2.2.0, 2.3.0
>
>
> The following example runs without error on Spark 2.0.x and 2.1.x but fails in the current
Spark master:
> {code}
> create temporary view foo (a, b) as values (cast(1 as bigint), 2), (cast(3 as bigint),
4);
> select a + b from foo group by a + b having (a + b) > 1 
> {code}
> The error is
> {code}
> Error in SQL statement: AnalysisException: cannot resolve '`a`' given input columns:
[(a + CAST(b AS BIGINT))]; line 1 pos 45;
> 'Filter (('a + 'b) > 1)
> +- Aggregate [(a#249243L + cast(b#249244 as bigint))], [(a#249243L + cast(b#249244 as
bigint)) AS (a + CAST(b AS BIGINT))#249246L]
>    +- SubqueryAlias foo
>       +- Project [col1#249241L AS a#249243L, col2#249242 AS b#249244]
>          +- LocalRelation [col1#249241L, col2#249242]
> {code}
> I think what's happening here is that the implicit cast is breaking things: if we change
the types so that both columns are integers then the analysis error disappears. Similarly,
adding explicit casts, as in
> {code}
> select a + cast(b as bigint) from foo group by a + cast(b as bigint) having (a + cast(b
as bigint)) > 1 
> {code}
> works so I'm pretty sure that the resolution problem is being introduced when the casts
are automatically added by the type coercion rule.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message