spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-7858) DataSourceStrategy.createPhysicalRDD should use output schema when performing row conversions, not relation schema
Date Tue, 26 May 2015 02:34:17 GMT

     [ https://issues.apache.org/jira/browse/SPARK-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-7858:
-----------------------------------

    Assignee: Josh Rosen  (was: Apache Spark)

> DataSourceStrategy.createPhysicalRDD should use output schema when performing row conversions,
not relation schema
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-7858
>                 URL: https://issues.apache.org/jira/browse/SPARK-7858
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.4.0
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> In {{DataSourceStrategy.createPhysicalRDD}}, we use the relation schema as the target
schema for converting incoming rows into Catalyst rows.  However, we should be using the output
schema instead, since our scan might be returning fewer columns due to partition pruning.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message