beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <>
Subject [jira] [Updated] (BEAM-1659) Improve UnboundedSourceShard structure
Date Mon, 03 Apr 2017 18:04:42 GMT


Thomas Groh updated BEAM-1659:
    Labels: starter  (was: )

> Improve UnboundedSourceShard structure
> --------------------------------------
>                 Key: BEAM-1659
>                 URL:
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-direct
>            Reporter: Thomas Groh
>            Priority: Minor
>              Labels: starter
> Use the 'OneOf' of "Active Reader" and "Optional<Checkpoint>", to make it obvious
how to obtain and use a reader.
> Maintain an ordered list of checkpoints that have not been finalized for a shard. Checkpoints
that can be resumed from can also be finalized, but otherwise the two are unrelated. This
also improves ease of never multi-finalizing a checkpoint. This gets more complicated with
retries, but must be considered with the associated work required to make UnboundedReadEvaluatorFactory
handle retries properly.

This message was sent by Atlassian JIRA

View raw message