flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Janani Chakkaradhari <janani.cs...@gmail.com>
Subject Re: UDP support in Streaming API
Date Thu, 26 Mar 2015 12:15:56 GMT
Hi Stephan,

Yes, you are right. I will try writing a custom data source as you
mentioned. Also, I still need to check the possibility of our system to use
Apache Kafka as a broker. Could it be possible for you to point out here
the downsides of using UDP as a source for streaming data?

Thanks,
Janani

On Wed, Mar 25, 2015 at 9:44 PM, Stephan Ewen <sewen@apache.org> wrote:

> Hi Janani!
>
> Do I understand you correctly in that you want a Flink stream source that
> receives UDP datagrams and turns them into Flink DataStream?
>
> Such a thing is not in there, yet. The interface to define custom data
> sources is rather simple, though, it should be possible to add something
> like this.
>
> Two things are not clear to me however:
>
> 1) Where are the datagrams sent to? That would need to be the point where
> the data source runs.
>
> 2) You need a way of turning the datagrams (which are just bytes) into
> records. Would that be hard-wired in your case?
>
> So, while probably possible, I would guess a UDP source has quite a few
> downsides over a message queue as a source. Using something like Kafka to
> communicate the source data is easier and better recoverable.
>
> Does your setup allow to put the data into Kafka and having Flink read the
> streams from Kafka?
>
> Greetings,
> Stephan
>  Am 25.03.2015 19:20 schrieb "Janani Chakkaradhari" <
> janani.cse05@gmail.com
> >:
>
> > ​Hi,
> >
> > Does Flink's streaming api has support for
> > ​ reading streams of data via Java UDP (DatagramSocket)? If so kindly
> > advise me in which release of Flink I can find it.
> >
> > Thanks,
> > Janani
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message