flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8538) Add a Kafka table source factory with JSON format support
Date Fri, 23 Feb 2018 16:32:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374579#comment-16374579
] 

ASF GitHub Bot commented on FLINK-8538:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5564#discussion_r170256878
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/descriptors/RowtimeValidator.scala
---
    @@ -77,58 +73,113 @@ class RowtimeValidator(val prefix: String = "") extends DescriptorValidator
{
     object RowtimeValidator {
     
       val ROWTIME = "rowtime"
    -
    -  // per rowtime properties
    -
    -  val ROWTIME_VERSION = "version"
    -  val TIMESTAMPS_TYPE = "timestamps.type"
    -  val TIMESTAMPS_TYPE_VALUE_FROM_FIELD = "from-field"
    -  val TIMESTAMPS_TYPE_VALUE_FROM_SOURCE = "from-source"
    -  val TIMESTAMPS_TYPE_VALUE_CUSTOM = "custom"
    -  val TIMESTAMPS_FROM = "timestamps.from"
    -  val TIMESTAMPS_CLASS = "timestamps.class"
    -  val TIMESTAMPS_SERIALIZED = "timestamps.serialized"
    -
    -  val WATERMARKS_TYPE = "watermarks.type"
    -  val WATERMARKS_TYPE_VALUE_PERIODIC_ASCENDING = "periodic-ascending"
    -  val WATERMARKS_TYPE_VALUE_PERIODIC_BOUNDING = "periodic-bounding"
    -  val WATERMARKS_TYPE_VALUE_FROM_SOURCE = "from-source"
    -  val WATERMARKS_TYPE_VALUE_CUSTOM = "custom"
    -  val WATERMARKS_CLASS = "watermarks.class"
    -  val WATERMARKS_SERIALIZED = "watermarks.serialized"
    -  val WATERMARKS_DELAY = "watermarks.delay"
    +  val ROWTIME_TIMESTAMPS_TYPE = "rowtime.timestamps.type"
    +  val ROWTIME_TIMESTAMPS_TYPE_VALUE_FROM_FIELD = "from-field"
    +  val ROWTIME_TIMESTAMPS_TYPE_VALUE_FROM_SOURCE = "from-source"
    +  val ROWTIME_TIMESTAMPS_TYPE_VALUE_CUSTOM = "custom"
    +  val ROWTIME_TIMESTAMPS_FROM = "rowtime.timestamps.from"
    +  val ROWTIME_TIMESTAMPS_CLASS = "rowtime.timestamps.class"
    +  val ROWTIME_TIMESTAMPS_SERIALIZED = "rowtime.timestamps.serialized"
    +
    +  val ROWTIME_WATERMARKS_TYPE = "rowtime.watermarks.type"
    +  val ROWTIME_WATERMARKS_TYPE_VALUE_PERIODIC_ASCENDING = "periodic-ascending"
    +  val ROWTIME_WATERMARKS_TYPE_VALUE_PERIODIC_BOUNDING = "periodic-bounding"
    +  val ROWTIME_WATERMARKS_TYPE_VALUE_FROM_SOURCE = "from-source"
    +  val ROWTIME_WATERMARKS_TYPE_VALUE_CUSTOM = "custom"
    +  val ROWTIME_WATERMARKS_CLASS = "rowtime.watermarks.class"
    +  val ROWTIME_WATERMARKS_SERIALIZED = "rowtime.watermarks.serialized"
    --- End diff --
    
    `rowtime.watermarks.custom.serialized`?


> Add a Kafka table source factory with JSON format support
> ---------------------------------------------------------
>
>                 Key: FLINK-8538
>                 URL: https://issues.apache.org/jira/browse/FLINK-8538
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table API &amp; SQL
>            Reporter: Timo Walther
>            Assignee: Xingcan Cui
>            Priority: Major
>
> Similar to CSVTableSourceFactory a Kafka table source factory for JSON should be added.
This issue includes improving the existing JSON descriptor with validation that can be used
for other connectors as well. It is up for discussion if we want to split the KafkaJsonTableSource
into connector and format such that we can reuse the format for other table sources as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message