spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "howie yu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-19512) codegen for compare structs fails
Date Tue, 08 May 2018 13:19:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467403#comment-16467403
] 

howie yu edited comment on SPARK-19512 at 5/8/18 1:18 PM:
----------------------------------------------------------

Hi

I think this  is similar issue, may not the same.

I have create a  project can re-produce this error

[https://github.com/howie/spark-codegen-error]

 S

 


was (Author: howieyu):
Hi


I think this  is similar issue, may not the same.

I have create a  project can re-produce this error

[https://github.com/howie/spark-codegen-error]

 

 

> codegen for compare structs fails
> ---------------------------------
>
>                 Key: SPARK-19512
>                 URL: https://issues.apache.org/jira/browse/SPARK-19512
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.0.0, 2.1.0
>            Reporter: Bogdan Raducanu
>            Assignee: Bogdan Raducanu
>            Priority: Major
>             Fix For: 2.1.1, 2.2.0
>
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
>       .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) as col2")
>       .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info]   Cause: java.util.concurrent.ExecutionException: java.lang.Exception: failed
to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 144,
Column 32: Expression "range_value" is not an rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10)
>     .selectExpr("named_struct('a', id, 'b', id) as col1", "named_struct('a',id+2, 'b',id+2)
as col2")
>     .filter($"col1" === $"col2").count
> {code}
> fails with 
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
>   at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
>   at scala.collection.immutable.List.apply(List.scala:84)
>   at org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message