flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From twalthr <...@git.apache.org>
Subject [GitHub] flink pull request #6323: [FLINK-8558] [FLINK-8866] [table] Finalize unified...
Date Fri, 13 Jul 2018 06:52:00 GMT
GitHub user twalthr opened a pull request:


    [FLINK-8558] [FLINK-8866] [table] Finalize unified table source/sink/format interfaces

    ## What is the purpose of the change
    This PR finalizes the efforts done in #6264 and #6201 for having unified interfaces for
table sources, table sinks, and table formats. It reduces code duplication and cleans up the
code base around factories.
    ## Brief change log
    - Introduction of `org.apache.table.factories.TableFactory` a common interface for factories
    - Introduction of `org.apache.table.factories.TableFormatFactory` a specific table factory
for formats
    - Specific factories for `StreamTableSource`, `StreamTableSink`, `BatchTableSource`, `BatchTableSink`,
`DeserializationSchema`, and `SerializationSchema`
    - Deprecation of old format-specific table sources (sinks will be deprecated in a follow-up
    - Possibility to register table source and sink under a common name (table type `both`
in SQL Client YAML)
    ## Verifying this change
    - Existing tests verify the implementation
    - Additional ITCases and unit tests have been added
    - (An end-to-end test will follow in a separate PR)
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): no
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes
      - The serializers: no
      - The runtime per-record code paths (performance sensitive): no
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: no
      - The S3 file system connector: no
    ## Documentation
      - Does this pull request introduce a new feature? yes
      - If yes, how is the feature documented? not documented

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/twalthr/flink UnifiedInterfacesFinal

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #6323
commit 980499f887d72ddf9a405c4ad200d0cab15d889c
Author: Timo Walther <twalthr@...>
Date:   2018-06-27T11:16:49Z

    [FLINK-8558] [table] Add unified format interfaces and separate formats from connectors
    This PR introduces a format discovery mechanism based on Java Service Providers. The general
`TableFormatFactory` is similar to the existing table source discovery mechanism. However,
it allows for arbirary format interfaces that might be introduced in the future. At the moment,
a connector can request configured instances of `DeserializationSchema` and `SerializationSchema`.
In the future we can add interfaces such as a `Writer` or `KeyedSerializationSchema` without
breaking backwards compatibility.
    This PR deprecates the existing strong coupling of connector and format for the Kafa table
sources and table source factories. It introduces descriptor-based alternatives.

commit 42a8a156d4e6f8f3d119c458350b6c897306fc48
Author: Shuyi Chen <shuyi@...>
Date:   2018-06-19T19:00:34Z

    [FLINK-8866] [table] Create unified interfaces to configure and instatiate TableSinks
    This closes #6201.

commit 311dc62e59c0e4146c094b73c21b979f31b2e1d9
Author: Timo Walther <twalthr@...>
Date:   2018-07-11T11:29:03Z

    Rename to TableFactory and move it to factories package

commit 1c581cba61ba321bb6de6a4d298a881840d11cfe
Author: Timo Walther <twalthr@...>
Date:   2018-07-11T11:46:31Z

    Refactor format factories

commit 5c6df7598d1f1c3c698ae9b6b35eb37d7fff8295
Author: Timo Walther <twalthr@...>
Date:   2018-07-12T06:35:00Z

    Unify table factories

commit 0cd7c44c006aba21c32d8785d17bfc3dbee03916
Author: Timo Walther <twalthr@...>
Date:   2018-07-12T07:05:50Z

    Move table type out of descriptors

commit 6b83f2e1c0e63147f049dc5389c5633077b789a4
Author: Timo Walther <twalthr@...>
Date:   2018-07-12T08:50:09Z

    Make source/sink factories environment-dependent

commit 4f1255fd003080f078afe6ef67ffa58f40ffec36
Author: Timo Walther <twalthr@...>
Date:   2018-07-12T18:48:45Z

    Clean up and simplify changes



View raw message