flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Max Michels (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1427) Configuration through environment variables
Date Fri, 23 Jan 2015 15:36:36 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289394#comment-14289394

Max Michels commented on FLINK-1427:

[~aalexandrov] I changed my mind a bit and actually agree with you. The configured state should
be transparent to the user. It's not transparent if the shell stores the state of Flink's
configuration (i.e. two shells may execute Flink differently).

[~uce] The environment variables already set would be deleted if the {{ConfigParser}} set
the environment variables. This defeats the purpose of setting the env variables in the first

I like [~rmetzger]'s idea. I'd like to have a flag for the start-local.sh or start-cluster.sh
to call the {{envToConf.sh}} script. Otherwise, this wouldn't be much easier then executing
{{ echo "config: value" >> flink-conf.yaml }}. Additionally, it could then also be logged
that environment variables were set!

> Configuration through environment variables
> -------------------------------------------
>                 Key: FLINK-1427
>                 URL: https://issues.apache.org/jira/browse/FLINK-1427
>             Project: Flink
>          Issue Type: Improvement
>          Components: Local Runtime
>         Environment: Deployment
>            Reporter: Max Michels
>            Priority: Minor
>              Labels: configuration, deployment
> Like Hadoop or Spark, etc. Flink should support configuration via shell environment variables.
In cluster setups, this makes things a lot easier because writing config files can be omitted.
Many automation tools (e.g. Google's bdutil) use (or abuse) this feature.
> For example, to set up the task manager heap size, we would run `export FLINK_TASKMANAGER_HEAP=4096`
before starting the task manager on a node to set the heap memory size to 4096MB.
> Environment variables should overwrite the regular config entries.

This message was sent by Atlassian JIRA

View raw message