beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Pouttu-Clarke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-91) Retractions
Date Tue, 21 Jun 2016 17:32:57 GMT

    [ https://issues.apache.org/jira/browse/BEAM-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342259#comment-15342259
] 

Matt Pouttu-Clarke commented on BEAM-91:
----------------------------------------

Yes, just wanted to be sure there was interest before documenting it.  FYI: it requires some
form of distributed configuration service such as etcd or zookeeper to keep track of in-process
change sessions.  Once the change sessions are done or "committed" (or time out), they are
cleared from the config service but can be obtained from logs for later replays.  Also, in
terms of granularity of change sessions, a large number of change sessions making very small
changes can cause problems for the design and should be throttled at the client side.  I'll
post a link here to the doco

> Retractions
> -----------
>
>                 Key: BEAM-91
>                 URL: https://issues.apache.org/jira/browse/BEAM-91
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Tyler Akidau
>            Assignee: Frances Perry
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> We still haven't added retractions to Beam, even though they're a core part of the model.
We should document all the necessary aspects (uncombine, reverting DoFn output with DoOvers,
sink integration, source-level retractions, etc), and then implement them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message