flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shijinkui <shijin...@huawei.com>
Subject [DISCUSS] add netty tcp/restful pushed source support
Date Mon, 26 Sep 2016 13:01:08 GMT
Hi, all

1.    In order to support end-to-end pushed source, I create FLINK-4630<https://issues.apache.org/jira/browse/FLINK-4630>.
I want to know whether is this idea worth?

When source stream get start, listen a provided tcp port, receive stream data from user data
This netty tcp source is keepping alive and end-to-end, that is from business system to flink
worker directly.

user app push ----->  netty server source of Flink

describe the source in detail below:

1.    source run as a netty tcp server
2.    user provide a tcp port, if the port is in used, increace the port number between 1024
to 65535. Source can parallel.
3.    callback the provided url to report the real port to listen
4.    user push streaming data to netty server, then collect the data to flink


Jinkui Shi

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message