flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9762) CoreOptions.TMP_DIRS wrongly managed on Yarn
Date Mon, 16 Jul 2018 17:02:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545487#comment-16545487
] 

ASF GitHub Bot commented on FLINK-9762:
---------------------------------------

Github user JTaky commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6284#discussion_r202756519
  
    --- Diff: flink-core/src/main/java/org/apache/flink/configuration/CoreOptions.java ---
    @@ -181,12 +181,17 @@
     	 * The config parameter defining the directories for temporary files, separated by
     	 * ",", "|", or the system's {@link java.io.File#pathSeparator}.
     	 */
    -	@Documentation.OverrideDefault("System.getProperty(\"java.io.tmpdir\")")
    +	@Documentation.OverrideDefault("'LOCAL_DIRS' on Yarn and '_FLINK_TMP_DIR' on Mesos.")
     	public static final ConfigOption<String> TMP_DIRS =
     		key("io.tmp.dirs")
     			.defaultValue(System.getProperty("java.io.tmpdir"))
     			.withDeprecatedKeys("taskmanager.tmp.dirs");
     
    +	/**
    +	 * String key, which says if variable `java.io.tmpdir` has been overridden for the cluster.
    +	 */
    +	public static final String TMP_DIRS_OVERRIDDEN = "io.tmp.dirs.overridden";
    --- End diff --
    
    done


> CoreOptions.TMP_DIRS wrongly managed on Yarn
> --------------------------------------------
>
>                 Key: FLINK-9762
>                 URL: https://issues.apache.org/jira/browse/FLINK-9762
>             Project: Flink
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.5.0
>            Reporter: Oleksandr Nitavskyi
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.5.2, 1.6.0
>
>
> The issue on Yarn is that it is impossible to have different LOCAL_DIRS on JobManager
and TaskManager, despite LOCAL_DIRS value depends on the container.
> The issue is that CoreOptions.TMP_DIRS is configured to the default value during JobManager
initialization and added to the configuration object. When TaskManager is launched the appropriate
configuration object is cloned with LOCAL_DIRS which makes sense only for Job Manager container.
When YARN container with TaskManager from his point of view CoreOptions.TMP_DIRS is always
equal either to path in flink.yml or to the or to the LOCAL_DIRS of Job Manager (default behaviour).
Is TaskManager’s container do not have an access to another folders, that folders allocated
by YARN TaskManager cannot be started.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message