spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Armbrust (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-5092) Selecting from a nested structure with SparkSQL should return a nested structure
Date Thu, 09 Jul 2015 00:27:05 GMT

     [ https://issues.apache.org/jira/browse/SPARK-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael Armbrust resolved SPARK-5092.
-------------------------------------
    Resolution: Won't Fix

> Selecting from a nested structure with SparkSQL should return a nested structure
> --------------------------------------------------------------------------------
>
>                 Key: SPARK-5092
>                 URL: https://issues.apache.org/jira/browse/SPARK-5092
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 1.2.0
>            Reporter: Brad Willard
>            Priority: Minor
>              Labels: pyspark, spark, sql
>
> When running a sparksql query like this (at least on a json dataset)
> select
>    rid,
>    meta_data.name
> from
>    a_table
> The rows returned lose the nested structure. I receive a row like
> Row(rid='123', name='delete')
> instead of
> Row(rid='123', meta_data=Row(name='data'))
> I personally think this is confusing especially when programmatically building and executing
queries and then parsing it to find your data in a new structure. I could understand how that's
less desirable in some situations, but you could get around it by supporting 'as'. If you
wanted to skip the nested structure simply write.
> select
>    rid,
>    meta_data.name as name
> from
>    a_table



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message