spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bjoern Toldbod (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-21691) Accessing canonicalized plan for query with limit throws exception
Date Mon, 28 Aug 2017 08:24:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-21691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143511#comment-16143511
] 

Bjoern Toldbod commented on SPARK-21691:
----------------------------------------

I have worked around the issue by not accessing the canonicalized logical plan, instead using
the logical plan itself.

Our application works with dynamic (user provided queries). 
We need to know which tables are referenced by a given query and we inspect the execution
plans in order to determine this.

I don't know of any alternative to inspecting the plans (other than writing my own sql-parser).

> Accessing canonicalized plan for query with limit throws exception
> ------------------------------------------------------------------
>
>                 Key: SPARK-21691
>                 URL: https://issues.apache.org/jira/browse/SPARK-21691
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.0
>            Reporter: Bjoern Toldbod
>
> Accessing the logical, canonicalized plan fails for queries with limits.
> The following demonstrates the issue:
> {code:java}
> val session = SparkSession.builder.master("local").getOrCreate()
> // This works
> session.sql("select * from (values 0, 1)").queryExecution.logical.canonicalized
> // This fails
> session.sql("select * from (values 0, 1) limit 1").queryExecution.logical.canonicalized
> {code}
> The message in the thrown exception is somewhat confusing (or at least not directly related
to the limit):
> "Invalid call to toAttribute on unresolved object, tree: *"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message