beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Davor Bonaci (JIRA)" <>
Subject [jira] [Commented] (BEAM-1691) Dynamic properties supported in PipelineOptions
Date Wed, 15 Mar 2017 01:53:41 GMT


Davor Bonaci commented on BEAM-1691:

[~mingmxu], can you say more please?

Generally speaking, runners may require different pipeline options. Options are provided by
individual runners and they have to be on the classpath to be able to set such options. If
users wants multiple runners on the classpath, this will just work -- options for multiple
runners can be set.

However -- users need to tweak at least one option: --runner. Tweaking any runner-specific
along the way isn't that different.

> Dynamic properties supported in PipelineOptions
> -----------------------------------------------
>                 Key: BEAM-1691
>                 URL:
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-core
>            Reporter: Xu Mingmin
>            Assignee: Davor Bonaci
> Usually the two lines to create a new Beam pipeline are:
> {code}
> Options options = PipelineOptionsFactory.fromArgs(args).withValidation().as(Options.class);
> Pipeline pipeline = Pipeline.create(options);
> {code} 
> As each runner has its own PipelineOptions, one piece of code is hardly to run on different
runners without code change, --as least Options needs to be updated.
> Dynamic property could be a choice, similar as
> {code}
> -D property1=value1 -D property2=value2 ...
> {code}

This message was sent by Atlassian JIRA

View raw message