flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-2393) Add a "stateless at-least-once" mode for streaming
Date Sun, 02 Aug 2015 15:20:04 GMT

     [ https://issues.apache.org/jira/browse/FLINK-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stephan Ewen closed FLINK-2393.
-------------------------------

> Add a "stateless at-least-once" mode for streaming
> --------------------------------------------------
>
>                 Key: FLINK-2393
>                 URL: https://issues.apache.org/jira/browse/FLINK-2393
>             Project: Flink
>          Issue Type: New Feature
>          Components: Streaming
>    Affects Versions: 0.10
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>             Fix For: 0.10
>
>
> Currently, the checkpointing mechanism provides "exactly once" guarantees. Part of that
is the step that temporarily "aligns" the data streams. This step increases the tuple latency
temporarily.
> By offering a version that does not provide "exactly-once", but only "at-least-once",
we can avoid the latency increase. For super-low-latency applications, that tolerate duplicates,
this may be an interesting option.
> To realize that, we would use a slightly modified version of the checkpointing algorithm.
Effectively, the streams would not be aligned, but tasks would only count the received barriers
and emit their own barrier as soon as the saw a barrier from all inputs.
> My feeling is that it makes not sense to implement state backups, when being concerned
with this super low latency. The mode would hence be a purely stateless at-least-once mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message