hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vinoyang (Jira)" <j...@apache.org>
Subject [jira] [Created] (HUDI-608) Implement a flink datastream execution context
Date Thu, 13 Feb 2020 14:35:00 GMT
vinoyang created HUDI-608:
-----------------------------

             Summary: Implement a flink datastream execution context
                 Key: HUDI-608
                 URL: https://issues.apache.org/jira/browse/HUDI-608
             Project: Apache Hudi (incubating)
          Issue Type: Sub-task
            Reporter: vinoyang
            Assignee: vinoyang


Currently {{HoodieWriteClient}} does something like `hoodieRecordRDD.map().sort()` internally..
if we want to support Flink DataStream as the object, then we need to somehow define an abstraction
like {{HoodieExecutionContext<T>}}  which will have a common set of map(T) -> T,
filter(), repartition() methods. There will be subclass like {{HoodieFlinkDataStreamExecutionContext<DataStream>}}
which will implement it in Flink specific ways and hand back the transformed T object.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message