flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: [DISCUSS] add netty tcp/restful pushed source support
Date Tue, 27 Sep 2016 08:54:15 GMT
I think that could be an interesting source. Two quick questions to move
forward

  - To keep the Flink code base from becoming too big (hard to maintain and
test) we started working with Apache Bahir as a project dedicated to
streaming connectors. Would that be a good target for the connector?

  - What are your thoughts on fault tolerance for that connector?

On Mon, Sep 26, 2016 at 3:01 PM, shijinkui <shijinkui@huawei.com> wrote:

> Hi, all
>
> 1.    In order to support end-to-end pushed source, I create FLINK-4630<
> https://issues.apache.org/jira/browse/FLINK-4630>. I want to know whether
> is this idea worth?
>
> -------------------
> When source stream get start, listen a provided tcp port, receive stream
> data from user data source.
> This netty tcp source is keepping alive and end-to-end, that is from
> business system to flink worker directly.
>
> user app push ----->  netty server source of Flink
>
> describe the source in detail below:
>
> 1.    source run as a netty tcp server
> 2.    user provide a tcp port, if the port is in used, increace the port
> number between 1024 to 65535. Source can parallel.
> 3.    callback the provided url to report the real port to listen
> 4.    user push streaming data to netty server, then collect the data to
> flink
>
>
> Thanks
>
> Jinkui Shi
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message