flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9762) CoreOptions.TMP_DIRS wrongly managed on Yarn
Date Mon, 16 Jul 2018 17:02:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545487#comment-16545487

ASF GitHub Bot commented on FLINK-9762:

Github user JTaky commented on a diff in the pull request:

    --- Diff: flink-core/src/main/java/org/apache/flink/configuration/CoreOptions.java ---
    @@ -181,12 +181,17 @@
     	 * The config parameter defining the directories for temporary files, separated by
     	 * ",", "|", or the system's {@link java.io.File#pathSeparator}.
    -	@Documentation.OverrideDefault("System.getProperty(\"java.io.tmpdir\")")
    +	@Documentation.OverrideDefault("'LOCAL_DIRS' on Yarn and '_FLINK_TMP_DIR' on Mesos.")
     	public static final ConfigOption<String> TMP_DIRS =
    +	/**
    +	 * String key, which says if variable `java.io.tmpdir` has been overridden for the cluster.
    +	 */
    +	public static final String TMP_DIRS_OVERRIDDEN = "io.tmp.dirs.overridden";
    --- End diff --

> CoreOptions.TMP_DIRS wrongly managed on Yarn
> --------------------------------------------
>                 Key: FLINK-9762
>                 URL: https://issues.apache.org/jira/browse/FLINK-9762
>             Project: Flink
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.5.0
>            Reporter: Oleksandr Nitavskyi
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.5.2, 1.6.0
> The issue on Yarn is that it is impossible to have different LOCAL_DIRS on JobManager
and TaskManager, despite LOCAL_DIRS value depends on the container.
> The issue is that CoreOptions.TMP_DIRS is configured to the default value during JobManager
initialization and added to the configuration object. When TaskManager is launched the appropriate
configuration object is cloned with LOCAL_DIRS which makes sense only for Job Manager container.
When YARN container with TaskManager from his point of view CoreOptions.TMP_DIRS is always
equal either to path in flink.yml or to the or to the LOCAL_DIRS of Job Manager (default behaviour).
Is TaskManager’s container do not have an access to another folders, that folders allocated
by YARN TaskManager cannot be started.

This message was sent by Atlassian JIRA

View raw message