flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyula.f...@gmail.com>
Subject Re: Thoughts About Streaming
Date Mon, 22 Jun 2015 19:55:48 GMT
Hi Aljoscha,

Thanks for the nice summary, this is a very good initiative.

I added some comments to the respective sections (where I didnt fully agree
:).).
At some point I think it would be good to have a public hangout session on
this, which could make a more dynamic discussion.

Cheers,
Gyula

Aljoscha Krettek <aljoscha@apache.org> ezt írta (időpont: 2015. jún. 22.,
H, 21:34):

> Hi,
> with people proposing changes to the streaming part I also wanted to throw
> my hat into the ring. :D
>
> During the last few months, while I was getting acquainted with the
> streaming system, I wrote down some thoughts I had about how things could
> be improved. Hopefully, they are in somewhat coherent shape now, so please
> have a look if you are interested in this:
>
> https://docs.google.com/document/d/1rSoHyhUhm2IE30o5tkR8GEetjFvMRMNxvsCfoPsW6_4/edit?usp=sharing
>
> This mostly covers:
>  - Timestamps assigned at sources
>  - Out-of-order processing of elements in window operators
>  - API design
>
> Please let me know what you think. Comment in the document or here in the
> mailing list.
>
> I have a PR in the makings that would introduce source timestamps and
> watermarks for keeping track of them. I also hacked a proof-of-concept of a
> windowing system that is able to process out-of-order elements using a
> FlatMap operator. (It uses panes to perform efficient pre-aggregations.)
>
> Cheers,
> Aljoscha
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message