beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Groh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-3102) Implement a BeamFnData Service
Date Thu, 26 Oct 2017 18:33:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Groh updated BEAM-3102:
------------------------------
    Labels: portability  (was: )

> Implement a BeamFnData Service
> ------------------------------
>
>                 Key: BEAM-3102
>                 URL: https://issues.apache.org/jira/browse/BEAM-3102
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-core
>            Reporter: Thomas Groh
>              Labels: portability
>
> The BeamFnData service provides a single API which streams elements to and from an SDK
harness. There should be a Java interface for Runners to interact with this API via Java methods.
> In general, the two directions should be handled as follows:
> Outbound elements (e.g. from the runner's perspective, from a Runner to an SDK harness)
should return some handle that consumes elements and can be completed. This is initially configured
with the destination that the elements will be consumed for.
> Inbound elements (e.g. from the runner's perspective, from an SDK harness to the runner)
should receive a consumer of those elements (e.g. write to some durable storage, to a stream
that groups elements, etc) and should produce some signal of the producer of those elements
having either produced all of the elements it is capable of, or completing exceptionally (and
will no longer produce additional elements)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message