spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng Lian (JIRA)" <>
Subject [jira] [Commented] (SPARK-16303) Update SQL examples and programming guide
Date Tue, 05 Jul 2016 14:45:11 GMT


Cheng Lian commented on SPARK-16303:

Sure, thanks for volunteering! Actually, I've started working on a branch aiming to replace
all hard-coded Spark SQL examples. I'll push it later so that you and/or other people can

To disambiguate, I'll use "example code" or "examples" to refer source files contained in
the {{examples}} sub-project, and "sample snippets" to refer snippets in the programming guide
web page.

The major problem here is that existing Spark SQL examples are quite limited. More over, examples
in different languages are inconsistent: Java and Python examples are largely consistent with
the programming guide, while the Scala example ({{RDDRelation.scala}}) only illustrates a
fraction of documented features (I haven't checked R examples since I'm not an R guy). Ideally,
we should achieve the following goals:

# All hard-coded sample snippets should be moved into source files under {{examples}} sub-project.
# Each source file under {{examples}} sub-project should be a self-contained Spark application.
(Currently, none of our examples need to be splitted into multiple files.)
# Examples in all languages should be consistent. Of course, if a feature is not available
in a language, we can just skip it (e.g. Datasets are not available in Python and R).

It's OK if you're not familiar with one or more languages, since we can split this work into
multiple PRs, each contains updates for one or more language bindings.

> Update SQL examples and programming guide
> -----------------------------------------
>                 Key: SPARK-16303
>                 URL:
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Documentation, Examples
>    Affects Versions: 2.0.0
>            Reporter: Cheng Lian
>            Assignee: Cheng Lian
> We need to update SQL examples code under the {{examples}} sub-project, and then replace
hard-coded snippets in the SQL programming guide with snippets automatically extracted from
actual source files.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message