beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1393) Update Flink Runner to Flink 1.2.0
Date Thu, 09 Feb 2017 16:47:42 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859777#comment-15859777
] 

Aljoscha Krettek commented on BEAM-1393:
----------------------------------------

That last part is actually a bit more complicated. (I saw what you did in your PR and you're
definitely going in the right direction.)

We have to differentiate between where we store side-input state and pushed-back events. The
reason for this is that there can be keyed operators that have state and for those we need
to ensure that the pushed back elements end up on the correct operator when restoring. Which
only happens for keyed state. For non-keyed state, we need to store the elements in state
that is not BROADCAST, they can just be reshuffled to any operator.

> Update Flink Runner to Flink 1.2.0
> ----------------------------------
>
>                 Key: BEAM-1393
>                 URL: https://issues.apache.org/jira/browse/BEAM-1393
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-flink
>            Reporter: Aljoscha Krettek
>            Assignee: Jingsong Lee
>
> When we update to 1.2.0 we can use the new internal Timer API that is available to Flink
operators: {{InternalTimerService}} and also use broadcast state to store side-input data.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message