From issues-return-177232-archive-asf-public=cust-asf.ponee.io@flink.apache.org Fri Jul 13 08:52:02 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id F03B9180626 for ; Fri, 13 Jul 2018 08:52:01 +0200 (CEST) Received: (qmail 52438 invoked by uid 500); 13 Jul 2018 06:52:00 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 52429 invoked by uid 99); 13 Jul 2018 06:52:00 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Jul 2018 06:52:00 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id AEAF9DFC43; Fri, 13 Jul 2018 06:52:00 +0000 (UTC) From: twalthr To: issues@flink.apache.org Reply-To: issues@flink.apache.org Message-ID: Subject: [GitHub] flink pull request #6323: [FLINK-8558] [FLINK-8866] [table] Finalize unified... Content-Type: text/plain Date: Fri, 13 Jul 2018 06:52:00 +0000 (UTC) GitHub user twalthr opened a pull request: https://github.com/apache/flink/pull/6323 [FLINK-8558] [FLINK-8866] [table] Finalize unified table source/sink/format interfaces ## What is the purpose of the change This PR finalizes the efforts done in #6264 and #6201 for having unified interfaces for table sources, table sinks, and table formats. It reduces code duplication and cleans up the code base around factories. ## Brief change log - Introduction of `org.apache.table.factories.TableFactory` a common interface for factories - Introduction of `org.apache.table.factories.TableFormatFactory` a specific table factory for formats - Specific factories for `StreamTableSource`, `StreamTableSink`, `BatchTableSource`, `BatchTableSink`, `DeserializationSchema`, and `SerializationSchema` - Deprecation of old format-specific table sources (sinks will be deprecated in a follow-up PR) - Possibility to register table source and sink under a common name (table type `both` in SQL Client YAML) ## Verifying this change - Existing tests verify the implementation - Additional ITCases and unit tests have been added - (An end-to-end test will follow in a separate PR) ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? not documented You can merge this pull request into a Git repository by running: $ git pull https://github.com/twalthr/flink UnifiedInterfacesFinal Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/6323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6323 ---- commit 980499f887d72ddf9a405c4ad200d0cab15d889c Author: Timo Walther Date: 2018-06-27T11:16:49Z [FLINK-8558] [table] Add unified format interfaces and separate formats from connectors This PR introduces a format discovery mechanism based on Java Service Providers. The general `TableFormatFactory` is similar to the existing table source discovery mechanism. However, it allows for arbirary format interfaces that might be introduced in the future. At the moment, a connector can request configured instances of `DeserializationSchema` and `SerializationSchema`. In the future we can add interfaces such as a `Writer` or `KeyedSerializationSchema` without breaking backwards compatibility. This PR deprecates the existing strong coupling of connector and format for the Kafa table sources and table source factories. It introduces descriptor-based alternatives. commit 42a8a156d4e6f8f3d119c458350b6c897306fc48 Author: Shuyi Chen Date: 2018-06-19T19:00:34Z [FLINK-8866] [table] Create unified interfaces to configure and instatiate TableSinks This closes #6201. commit 311dc62e59c0e4146c094b73c21b979f31b2e1d9 Author: Timo Walther Date: 2018-07-11T11:29:03Z Rename to TableFactory and move it to factories package commit 1c581cba61ba321bb6de6a4d298a881840d11cfe Author: Timo Walther Date: 2018-07-11T11:46:31Z Refactor format factories commit 5c6df7598d1f1c3c698ae9b6b35eb37d7fff8295 Author: Timo Walther Date: 2018-07-12T06:35:00Z Unify table factories commit 0cd7c44c006aba21c32d8785d17bfc3dbee03916 Author: Timo Walther Date: 2018-07-12T07:05:50Z Move table type out of descriptors commit 6b83f2e1c0e63147f049dc5389c5633077b789a4 Author: Timo Walther Date: 2018-07-12T08:50:09Z Make source/sink factories environment-dependent commit 4f1255fd003080f078afe6ef67ffa58f40ffec36 Author: Timo Walther Date: 2018-07-12T18:48:45Z Clean up and simplify changes ---- ---