flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From twal...@apache.org
Subject flink git commit: [FLINK-6745] [table] [docs] Updated Table API / SQL docs: Overview
Date Tue, 06 Jun 2017 12:58:46 GMT
Repository: flink
Updated Branches:
  refs/heads/tableDocs cd329420e -> ca58e6d35

[FLINK-6745] [table] [docs] Updated Table API / SQL docs: Overview

This closes #4013.

Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/ca58e6d3
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/ca58e6d3
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/ca58e6d3

Branch: refs/heads/tableDocs
Commit: ca58e6d357eeeb6f34423ec3b576213015940d2f
Parents: cd32942
Author: Fabian Hueske <fhueske@apache.org>
Authored: Fri May 26 14:47:35 2017 +0100
Committer: twalthr <twalthr@apache.org>
Committed: Tue Jun 6 14:57:43 2017 +0200

 docs/dev/tableApi.md | 59 ++++++++++++++++++++++++-----------------------
 1 file changed, 30 insertions(+), 29 deletions(-)

diff --git a/docs/dev/tableApi.md b/docs/dev/tableApi.md
index c1c02f1..f7b13f0 100644
--- a/docs/dev/tableApi.md
+++ b/docs/dev/tableApi.md
@@ -25,32 +25,16 @@ specific language governing permissions and limitations
 under the License.
-**Table API and SQL are experimental features**
+Apache Flink features two relational APIs - the Table API and SQL - for unified stream and
batch processing. The Table API is a language-integrated query API for Scala and Java that
allows the composition of queries from relational operators such as selection, filter, and
join in a very intuitive way. Flink's SQL support is based on [Apache Calcite](https://calcite.apache.org)
which implements the SQL standard. Queries specified in either interface have the same semantics
and specify the same result regardless whether the input is a batch input (DataSet) or a stream
input (DataStream).
-The Table API is a SQL-like expression language for relational stream and batch processing
that can be easily embedded in Flink's DataSet and DataStream APIs (Java and Scala).
-The Table API and SQL interface operate on a relational `Table` abstraction, which can be
created from external data sources, or existing DataSets and DataStreams. With the Table API,
you can apply relational operators such as selection, aggregation, and joins on `Table`s.
+The Table API and the SQL interfaces are tightly integrated with each other as well as Flink's
DataStream and DataSet APIs. You can easily switch between all APIs and libraries which build
upon the APIs. For instance, you can extract patterns from a DataStream using the [CEP library]({{
site.baseurl }}/dev/libs/cep.html) and later use the Table API to analyze the patterns, or
you might scan, filter, and aggregate a batch table using a SQL query before running a [Gelly
graph algorithm]({{ site.baseurl }}/dev/libs/gelly) on the preprocessed data.
-`Table`s can also be queried with regular SQL, as long as they are registered (see [Registering
Tables](#registering-tables)). The Table API and SQL offer equivalent functionality and can
be mixed in the same program. When a `Table` is converted back into a `DataSet` or `DataStream`,
the logical plan, which was defined by relational operators and SQL queries, is optimized
using [Apache Calcite](https://calcite.apache.org/) and transformed into a `DataSet` or `DataStream`
-**TODO: Check, update, and add**
-* What are the Table API / SQL
-  * Relational APIs
-  * Unified APIs for batch and streaming
-    * Semantics are the same
-    * But not all operations can be efficiently mapped to streams
-  * Table API: language-integrated queries (LINQ) in Scala and Java
-  * SQL: Standard SQL
-**Please notice: Not all operations are supported by all four combinations of Stream/Batch
and TableAPI/SQL.**
-* This will be replaced by the TOC
+**Please note that the Table API and SQL are not yet feature complete and are being active
developed. Not all operations are supported by every combination of \[Table API, SQL\] and
\[stream, batch\] input.**
-The Table API and SQL are part of the *flink-table* Maven project.
+The Table API and SQL are bundled in the `flink-table` Maven artifact. 
 The following dependency must be added to your project in order to use the Table API and
 {% highlight xml %}
@@ -61,21 +45,38 @@ The following dependency must be added to your project in order to use
the Table
 {% endhighlight %}
-*Note: The Table API is currently not part of the binary distribution. See linking with it
for cluster execution [here]({{ site.baseurl }}/dev/linking.html).*
+In addition, you need to add a dependency for either Flink's Scala batch or streaming API.
For a batch query you need to add:
+{% highlight xml %}
+  <groupId>org.apache.flink</groupId>
+  <artifactId>flink-scala{{ site.scala_version_suffix }}</artifactId>
+  <version>{{site.version }}</version>
+{% endhighlight %}
+For a streaming query you need to add:
+{% highlight xml %}
+  <groupId>org.apache.flink</groupId>
+  <artifactId>flink-streaming-scala{{ site.scala_version_suffix }}</artifactId>
+  <version>{{site.version }}</version>
+{% endhighlight %}
-**TODO: Rework and add:**
-* Project dependencies (flink-table + flink-scala or flink-streaming-scala)
-* Copy `./opt/flink-table.jar` to `./lib`
+**Note:** Due to an issue in Apache Calcite, which prevents the user classloaders from being
garbage-collected, we do *not* recommend building a fat-jar that includes the `flink-table`
dependency. Instead, we recommend configuring Flink to include the `flink-table` dependency
in the system classloader. This can be done by copying the `flink-table.jar` file from the
`./opt` folder to the `./lib` folder. See [these instructions]({{ site.baseurl }}/dev/linking.html)
for further details.
 {% top %}
 Where to go next?
-* [Concepts & Common API]({{ site.baseurl }}/dev/table/common.html): Share concepts and
API of the Table API and SQL.
-* [Table API]({{ site.baseurl }}/dev/table/tableapi.html): Supported Operations and API for
the Table API
-* [SQL]({{ site.baseurl }}/dev/table/sql.html): Supported Operations and Syntax for SQL
-* [Table Sources & Sinks]({{ site.baseurl }}/dev/table/sourceSinks.html): Ingestion and
emission of tables.
-* [User-Defined Functions]({{ site.baseurl }}/dev/table/udfs.html): Defintion and usage of
user-defined functions.
+* [Concepts & Common API]({{ site.baseurl }}/dev/table/common.html): Shared concepts
and APIs of the Table API and SQL.
+* [Streaming Table API & SQL]({{ site.baseurl }}/dev/table/streaming.html): Streaming-specific
documentation for the Table API or SQL such as configuration of time attributes and handling
of updating results.
+* [Table API]({{ site.baseurl }}/dev/table/tableapi.html): Supported operations and API for
the Table API.
+* [SQL]({{ site.baseurl }}/dev/table/sql.html): Supported operations and syntax for SQL
+* [Table Sources & Sinks]({{ site.baseurl }}/dev/table/sourceSinks.html): Reading tables
from and emitting tables to external storage systems.
+* [User-Defined Functions]({{ site.baseurl }}/dev/table/udfs.html): Definition and usage
of user-defined functions.
 {% top %}
\ No newline at end of file

View raw message