beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (BEAM-551) Support Dynamic PipelineOptions
Date Mon, 15 Aug 2016 14:37:20 GMT


ASF GitHub Bot commented on BEAM-551:

GitHub user sammcveety opened a pull request:

    [BEAM-551] Add ValueProvider class

    Hi @lukecwik , can you please take a look?
    Currently implemented for ValueProvider<String> only.  Tests demonstrate intended
functionality, once the questions below are answered:
    - What's the best way to avoid threading issues with the static options in test?
    - Is it reasonable to continue down the road of special-casing getter/setter validation
for this class?
    - What would be the preferred way to annotate T for ValueProvider<T> to work around
type erasure?

You can merge this pull request into a Git repository by running:

    $ git pull sgmc/valueprovider

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #827
commit 867a080b98f585db2687d9aaf7d488882de8c813
Author: sammcveety <>
Date:   2016-08-14T00:45:58Z

    Merge pull request #1 from apache/master
    Update my fork.

commit bb7aea8db07b3a7ae988f75888716dd865e07e08
Author: sammcveety <>
Date:   2016-08-14T00:48:18Z


commit 5330800d9d5b54b897c53cb8ea40c6331fe3c36a
Author: sammcveety <>
Date:   2016-08-14T00:48:54Z


commit 9b9b17c2338612b4030d1c4a36f8fb4c5c179c2e
Author: sammcveety <>
Date:   2016-08-14T00:50:56Z


commit b54832bd0c28691d1e6087b4901fb6ae11ebd2c9
Author: sammcveety <>
Date:   2016-08-14T00:54:33Z


commit e2d16a7ad66676a099d4160ecd4c31c369acd46e
Author: sammcveety <>
Date:   2016-08-14T00:56:41Z


commit afbe8157f711fca00ed37afbe461edd4ac78e30f
Author: sammcveety <>
Date:   2016-08-14T16:34:23Z


commit e111045ae1125ed49ea1f26cdde31b3c1583ed2d
Author: sammcveety <>
Date:   2016-08-14T16:58:29Z


commit 0bff463a71c4f4b3ad13edd924fd03226a915a61
Author: sammcveety <>
Date:   2016-08-14T20:11:49Z

    Add tests.

commit c4f3acc427b828f0f0254f83e84690b14c0a0f98
Author: sammcveety <>
Date:   2016-08-14T21:42:56Z

    Iterations on tests.

commit f19435b688fed41d873b85ac7750d923025256a9
Author: sammcveety <>
Date:   2016-08-15T01:32:37Z

    Proposal for how to handle getters/setters.


> Support Dynamic PipelineOptions
> -------------------------------
>                 Key: BEAM-551
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Sam McVeety
>            Assignee: Frances Perry
>            Priority: Minor
> During the graph construction phase, the given SDK generates an initial
> execution graph for the program.  At execution time, this graph is
> executed, either locally or by a service.  Currently, Beam only supports
> parameterization at graph construction time.  Both Flink and Spark supply
> functionality that allows a pre-compiled job to be run without SDK
> interaction with updated runtime parameters.
> In its current incarnation, Dataflow can read values of PipelineOptions at
> job submission time, but this requires the presence of an SDK to properly
> encode these values into the job.  We would like to build a common layer
> into the Beam model so that these dynamic options can be properly provided
> to jobs.
> Please see
> for the high-level model, and
> for
> the specific API proposal.

This message was sent by Atlassian JIRA

View raw message