flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maximilian Michels (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (FLINK-4079) YARN properties file used for per-job cluster
Date Wed, 15 Jun 2016 17:02:09 GMT

     [ https://issues.apache.org/jira/browse/FLINK-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Maximilian Michels reopened FLINK-4079:
      Assignee: Maximilian Michels

I see what you mean now :) 

The issue is that the Yarn properties file is loaded regardless of whether "-m yarn-cluster"
is specified on the command-line. This loads the dynamic properties from the Yarn properties
file and applies all configuration of the running (session) cluster cluster to the to-be-created
cluster. This is not expected behavior.

> YARN properties file used for per-job cluster
> ---------------------------------------------
>                 Key: FLINK-4079
>                 URL: https://issues.apache.org/jira/browse/FLINK-4079
>             Project: Flink
>          Issue Type: Bug
>          Components: Command-line client
>    Affects Versions: 1.0.3
>            Reporter: Ufuk Celebi
>            Assignee: Maximilian Michels
>             Fix For: 1.1.0, 1.0.4
> YARN per job clusters (flink run -m yarn-cluster) rely on the hidden YARN properties
file, which defines the container configuration. This can lead to unexpected behaviour, because
the per-job-cluster configuration is merged  with the YARN properties file (or used as only
configuration source).
> A user ran into this as follows:
> - Create a long-lived YARN session with HA (creates a hidden YARN properties file)
> - Submits standalone batch jobs with a per job cluster (flink run -m yarn-cluster). The
batch jobs get submitted to the long lived HA cluster, because of the properties file.
> [~mxm] Am I correct in assuming that this is only relevant for the 1.0 branch and will
be fixed with the client refactoring you are working on?

This message was sent by Atlassian JIRA

View raw message