spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shivaram Venkataraman (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-12629) SparkR: DataFrame's saveAsTable method has issues with the signature and HiveContext
Date Fri, 22 Jan 2016 18:36:40 GMT

     [ https://issues.apache.org/jira/browse/SPARK-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shivaram Venkataraman updated SPARK-12629:
------------------------------------------
    Assignee: Narine Kokhlikyan

> SparkR: DataFrame's saveAsTable method has issues with the signature and HiveContext

> -------------------------------------------------------------------------------------
>
>                 Key: SPARK-12629
>                 URL: https://issues.apache.org/jira/browse/SPARK-12629
>             Project: Spark
>          Issue Type: Bug
>          Components: SparkR
>            Reporter: Narine Kokhlikyan
>            Assignee: Narine Kokhlikyan
>
> There are several issues with the DataFrame's saveAsTable method in SparkR. Here is a
summary of some of them. Hope this will help to fix the issues.
> 1. According to SparkR's saveAsTable(...) documentation, we can call the "saveAsTable(df,
"myfile")" in order to store the dataframe.
> However, this signature isn't working. It seems that "source" and "mode" are forced according
to signature.
> 2. Within the method saveAsTable(...) it tries to retrieve the SQL context and tries
to create/initialize source as parquet, but this is also failing because the context has to
be Hive Context. Based on the error messages I see.
> 3. In general the method fails when I try to call it with sqlContext
> 4. Also, it seems that SQL DataFrame.saveAsTable is deprecated, we could use df.write.saveAsTable(...)
instead ...
> [~shivaram] [~sunrui] [~felixcheung]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message