beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rafal Wojdyla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1458) Checkpoint support in Beam
Date Fri, 10 Feb 2017 21:32:42 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861876#comment-15861876
] 

Rafal Wojdyla commented on BEAM-1458:
-------------------------------------

[~aljoscha] thanks for the comment. Correct - in this case we are talking about "scalding
like" checkpoints, even if one can't use Flink snapshots like that. I'm not too worried about
support in all runners, no all runners have to immediately implement this feature.

> Checkpoint support in Beam
> --------------------------
>
>                 Key: BEAM-1458
>                 URL: https://issues.apache.org/jira/browse/BEAM-1458
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model
>    Affects Versions: 0.5.0
>            Reporter: Rafal Wojdyla
>            Assignee: Rafal Wojdyla
>              Labels: features
>
> Beam could support checkpoints - similar to:
>  * flink's snapshots
>  * scalding's checkpoints
> Checkpoint should provides a simple mechanism to read and write intermediate results.
It would be useful for debugging one part of a long flow, when you would otherwise have to
run many steps to get to the one you care about.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message