spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "pin_zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-11748) Result is null after alter column name of table stored as Parquet
Date Thu, 19 Nov 2015 06:56:11 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013030#comment-15013030
] 

pin_zhang commented on SPARK-11748:
-----------------------------------

Apache hive 0.14 has added Support for Parquet Column Rename https://issues.apache.org/jira/browse/HIVE-6938
That doesn't work in spark hive


> Result is null after alter column name of table stored as Parquet 
> ------------------------------------------------------------------
>
>                 Key: SPARK-11748
>                 URL: https://issues.apache.org/jira/browse/SPARK-11748
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.5.1
>            Reporter: pin_zhang
>
> 1. Test with the following code
>     hctx.sql(" create table " + table + " (id int, str string) STORED AS PARQUET ")
>     val df = hctx.jsonFile("g:/vip.json")
>     df.write.format("parquet").mode(SaveMode.Append).saveAsTable(table)
>     hctx.sql(" select * from " + table).show()
>     // alter table
>     val alter = "alter table " + table + " CHANGE id i_d int "
>     hctx.sql(alter)
>  
>     hctx.sql(" select * from " + table).show()
> 2. Result
> after change table column name, data in null for the changed column
> Result before alter table
> +---+---+
> | id|str|
> +---+---+
> |  1| s1|
> |  2| s2|
> +---+---+
> Result after alter table
> +----+---+
> | i_d|str|
> +----+---+
> |null| s1|
> |null| s2|
> +----+---+



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message