beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henning Rohde (JIRA)" <>
Subject [jira] [Updated] (BEAM-3102) Implement a BeamFnData Service
Date Fri, 27 Oct 2017 21:13:00 GMT


Henning Rohde updated BEAM-3102:
    Issue Type: Sub-task  (was: New Feature)
        Parent: BEAM-2899

> Implement a BeamFnData Service
> ------------------------------
>                 Key: BEAM-3102
>                 URL:
>             Project: Beam
>          Issue Type: Sub-task
>          Components: runner-core
>            Reporter: Thomas Groh
>              Labels: portability
> The BeamFnData service provides a single API which streams elements to and from an SDK
harness. There should be a Java interface for Runners to interact with this API via Java methods.
> In general, the two directions should be handled as follows:
> Outbound elements (e.g. from the runner's perspective, from a Runner to an SDK harness)
should return some handle that consumes elements and can be completed. This is initially configured
with the destination that the elements will be consumed for.
> Inbound elements (e.g. from the runner's perspective, from an SDK harness to the runner)
should receive a consumer of those elements (e.g. write to some durable storage, to a stream
that groups elements, etc) and should produce some signal of the producer of those elements
having either produced all of the elements it is capable of, or completing exceptionally (and
will no longer produce additional elements)

This message was sent by Atlassian JIRA

View raw message