flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5253) Remove special treatment of "dynamic properties"
Date Wed, 05 Jul 2017 15:14:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074913#comment-16074913
] 

ASF GitHub Bot commented on FLINK-5253:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3356#discussion_r125670830
  
    --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java
---
    @@ -711,8 +701,18 @@ public FileVisitResult preVisitDirectory(java.nio.file.Path dir,
BasicFileAttrib
     		LocalResource flinkConf = Records.newRecord(LocalResource.class);
     		Path remotePathJar =
     			Utils.setupLocalResource(fs, appId.toString(), flinkJarPath, appMasterJar, fs.getHomeDirectory());
    -		Path remotePathConf =
    -			Utils.setupLocalResource(fs, appId.toString(), flinkConfigurationPath, flinkConf,
fs.getHomeDirectory());
    +
    +		final org.apache.flink.configuration.Configuration dynamicProperties = GlobalConfiguration.getDynamicProperties();
    --- End diff --
    
    Can't we pass this configuration into the `ClusterDescriptor` differently? What about
reading it when creating the descriptor and then passing it as a constructor parameter? Even
better, we could read the global configuration, then parse the dynamic properties, adding
them to the read config and then creating the descriptor.


> Remove special treatment of "dynamic properties"
> ------------------------------------------------
>
>                 Key: FLINK-5253
>                 URL: https://issues.apache.org/jira/browse/FLINK-5253
>             Project: Flink
>          Issue Type: Sub-task
>          Components: YARN
>         Environment: {{flip-6}} feature branch
>            Reporter: Stephan Ewen
>              Labels: flip-6
>
> The YARN client accepts configuration keys as command line parameters.
> Currently these are send to the AppMaster and TaskManager as "dynamic properties", encoded
in a special way via environment variables.
> The mechanism is quite fragile. We should simplify it:
>   - The YARN client takes the local {{flink-conf.yaml}} as the base.
>   - It overwrite config entries with command line properties when preparing the configuration
to be shipped to YARN container processes (JM / TM)
>   - No additional handling neccessary



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message