beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Cwik (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1348) Model the Fn Api
Date Thu, 18 May 2017 22:47:04 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016590#comment-16016590
] 

Luke Cwik commented on BEAM-1348:
---------------------------------

Beam Fn API overview:
https://s.apache.org/beam-fn-api 

And for those of you who have been following this thread and contributors focusing on Runner
integration with Apache Beam:
* How to process a bundle: https://s.apache.org/beam-fn-api-processing-a-bundle
* How to send and receive data: https://s.apache.org/beam-fn-api-send-and-receive-data

If you want to dive deeper, you should look at:
* Runner API Protobuf: https://github.com/apache/beam/blob/master/sdks/common/runner-api/src/main/proto/beam_runner_api.proto
* Fn API Protobuf: https://github.com/apache/beam/blob/master/sdks/common/fn-api/src/main/proto/beam_fn_api.proto
* Java SDK Harness: https://github.com/apache/beam/tree/master/sdks/java/harness
* Python SDK Harness: https://github.com/apache/beam/tree/master/sdks/python/apache_beam/runners/worker


Next I'm planning on talking about Beam Fn State API and will need help from Runner contributors
to talk about caching semantics and key spaces and whether the integrations mesh well with
current Runner implementations. The State API is meant to support user state, side inputs,
and re-iteration for large values produced by GroupByKey.

> Model the Fn Api
> ----------------
>
>                 Key: BEAM-1348
>                 URL: https://issues.apache.org/jira/browse/BEAM-1348
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model-fn-api
>            Reporter: Luke Cwik
>            Assignee: Luke Cwik
>
> Create a proto representation of the services and data types required to execute the
Fn Api.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message