spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (JIRA)" <>
Subject [jira] [Commented] (SPARK-20055) Documentation for CSV datasets in SQL programming guide
Date Mon, 22 Oct 2018 16:22:00 GMT


Dongjoon Hyun commented on SPARK-20055:

This looks too old. Are we working on this now?

> Documentation for CSV datasets in SQL programming guide
> -------------------------------------------------------
>                 Key: SPARK-20055
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Documentation
>    Affects Versions: 2.2.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
> I guess things commonly used and important are documented there rather than documenting
everything and every option in the programming guide -
> It seems JSON datasets
are documented whereas CSV datasets are not. 
> Nowadays, they are pretty similar in APIs and options. Some options are notable for both,
In particular, ones such as {{wholeFile}}. Moreover, several options such as {{inferSchema}}
and {{header}} are important in CSV that affect the type/column name of data.
> In that sense, I think we might better document CSV datasets with some examples too because
I believe reading CSV is pretty much common use cases.
> Also, I think we could also leave some pointers for options of API documentations for
both (rather than duplicating the documentation).
> So, my suggestion is,
> - Add CSV Datasets section.
> - Add links for options for both JSON and CSV that point each API documentation
> - Fix trivial minor fixes together in both sections.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message