beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <>
Subject [jira] [Commented] (BEAM-2591) Python shim for submitting to FlinkRunner
Date Fri, 09 Feb 2018 00:23:00 GMT


Kenneth Knowles commented on BEAM-2591:

I think probably best to get feedback on idiomatic uses from [~robertwb], [~altay], [~jgrier],

Basically I imagine a user will
1. test bounded smoke test with the Python direct runner
2. try the same on a local flink (or whatever is a one-liner to launch and teardown) and also
some unbounded data at that point
3. go big by:
    a. point the pipeline at a production endpoint where they don't know or care what tech
is behind it
    b. or launch their pipeline main alongside flink as a complete app via k8s or a bespoke
    c. ???

This ticket had been filed for 2 with the idea that it would be reusable for 3 but actually
I think it should just be split into separate tasks and this focus on 2.

> Python shim for submitting to FlinkRunner
> -----------------------------------------
>                 Key: BEAM-2591
>                 URL:
>             Project: Beam
>          Issue Type: Sub-task
>          Components: runner-flink, sdk-py-core
>            Reporter: Kenneth Knowles
>            Priority: Major
>              Labels: portability
> Whatever the result of, Python users will need to be
able to pass --runner=FlinkRunner and have it work.

This message was sent by Atlassian JIRA

View raw message