spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yin Huai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-3641) Correctly populate SparkPlan.currentContext
Date Mon, 22 Sep 2014 16:06:33 GMT

    [ https://issues.apache.org/jira/browse/SPARK-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143347#comment-14143347
] 

Yin Huai commented on SPARK-3641:
---------------------------------

[~marmbrus] Can we populate SparkPlan.currentContext in the constructor of SQLContext instead
of populate it every time before using ExistingRDD?

> Correctly populate SparkPlan.currentContext
> -------------------------------------------
>
>                 Key: SPARK-3641
>                 URL: https://issues.apache.org/jira/browse/SPARK-3641
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.1.0
>            Reporter: Yin Huai
>            Priority: Critical
>
> After creating a new SQLContext, we need to populate SparkPlan.currentContext before
we create any SparkPlan. Right now, only SQLContext.createSchemaRDD populate SparkPlan.currentContext.
SQLContext.applySchema is missing this call and we can have NPE as described in http://qnalist.com/questions/5162981/spark-sql-1-1-0-npe-when-join-two-cached-table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message