flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Design documents for consolidated DataStream API
Date Tue, 07 Jul 2015 13:35:41 GMT
Hi,
I just noticed that we don't have anything about how iterations and
timestamps/watermarks should interact.

Cheers,
Aljoscha

On Mon, 6 Jul 2015 at 23:56 Stephan Ewen <sewen@apache.org> wrote:

> Hi all!
>
> As many of you know, there are a ongoing efforts to consolidate the
> streaming API for the next release, and then graduate it (from beta
> status).
>
> In the process of this consolidation, we want to achieve the following
> goals.
>
>  - Make the code more robust and simplify it in parts
>
>  - Clearly define the semantics of the constructs.
>
>  - Prepare it for support of more advanced concepts, like partitionable
> state, and event time.
>
>  - Cut support for certain corner cases that were prototyped, but turned
> out to be not efficiently doable
>
>
> Based on prior discussions on the mailing list, Aljoscha and me drafted the
> design documents below, which outline how the consolidated API would like.
> We focused in constructs, time, and window semantics.
>
>
> Design document on how to restructure the Streaming API:
>
> https://cwiki.apache.org/confluence/display/FLINK/Streams+and+Operations+on+Streams
>
> Design document on definitions of time, order, and the resulting semantics:
> https://cwiki.apache.org/confluence/display/FLINK/Time+and+Order+in+Streams
>
>
>
> Note: The design of the interfaces and concepts for advanced state in
> functions is not in here. That is part of a separate design discussion and
> orthogonal to the designs drafted here.
>
>
> Please have a look and voice questions and concerns. Since we should not
> break the streaming API more than once, we should make sure this
> consolidation brings it into the shape we want it to be in.
>
>
> Greetings,
> Stephan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message