flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: Are there pipeline API's for ETL?
Date Sat, 11 Jan 2020 05:39:37 GMT
Hi Vino,

Another use case would be I want to build a dag of batch sources, sinks and transforms and
I want to schedule the jobs periodically. One can say similar to airflow but a Flink api would
be lot better!

Sent from my iPhone

> On Jan 10, 2020, at 6:42 PM, vino yang <yanghua1127@gmail.com> wrote:
> 
> 
> Hi kant,
> 
> Can you provide more context about your question? What do you mean about "pipeline API"?
> 
> IMO, you can build an ETL pipeline via composing several Flink transform APIs. About
choosing which transform APIs, it depends on your business logic. 
> 
> Here are the generic APIs list.[1]
> 
> Best,
> Vino
> 
> [1]: https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/index.html
> 
> kant kodali <kanth909@gmail.com> 于2020年1月11日周六 上午9:06写道:
>> Hi All,
>> 
>> I am wondering if there are pipeline API's for ETL?
>> 
>> Thanks!
>> 
>> 

Mime
View raw message