beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (BEAM-2600) Python SDK harness container
Date Tue, 10 Oct 2017 21:30:00 GMT


ASF GitHub Bot commented on BEAM-2600:

Github user asfgit closed the pull request at:

> Python SDK harness container
> ----------------------------
>                 Key: BEAM-2600
>                 URL:
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-py-core
>            Reporter: Kenneth Knowles
>            Assignee: Henning Rohde
>              Labels: portability
> In order to build a pipeline that invokes a Python UDF, we need to be able to construct
something like this:
> {code}
> SdkFunctionSpec {
>   environment = <python SDK harness>,
>   spec = {
>     urn = <python SDK pickled DoFn>,
>     data = <pickled DoFn>
>   }
> }
> {code}
> I could be out of date, but based on a couple of conversations I do not know that there
exists anything we can put for "<python SDK harness>" today. For prototyping, it could
be just a symbol that runners have to know. But eventually it should be something that runners
can instantiate without knowing anything about the SDK that put it there. I imagine it may
encompass "custom containers" eventually, though that doesn't block anything immediately.
> Update 9/11/2017: this issue will track the first portable SDK harness as per,
incl shared boot code and maven integration to build Go and Docker container images.

This message was sent by Atlassian JIRA

View raw message