spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yanbo Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-8000) SQLContext.read.load() should be able to auto-detect input data
Date Mon, 21 Sep 2015 03:55:04 GMT

    [ https://issues.apache.org/jira/browse/SPARK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900172#comment-14900172
] 

Yanbo Liang commented on SPARK-8000:
------------------------------------

[~rxin] I will work on it.
I agree to make Spark SQL write an output metadata file.
But if the data is produced by other framework which did not have such metadata, it will not
work well. Does this in line with expectation?

> SQLContext.read.load() should be able to auto-detect input data
> ---------------------------------------------------------------
>
>                 Key: SPARK-8000
>                 URL: https://issues.apache.org/jira/browse/SPARK-8000
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Reynold Xin
>
> If it is a parquet file, use parquet. If it is a JSON file, use JSON. If it is an ORC
file, use ORC. If it is a CSV file, use CSV.
> Maybe Spark SQL can also write an output metadata file to specify the schema & data
source that's used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message