flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8475) Move remaining sections to generated tables
Date Mon, 12 Feb 2018 16:40:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361035#comment-16361035

ASF GitHub Bot commented on FLINK-8475:

Github user zentol commented on a diff in the pull request:

    --- Diff: docs/ops/config.md ---
    @@ -408,38 +408,7 @@ of the JobManager, because the same ActorSystem is used. Its not
possible to use
     ### YARN
    -- `containerized.heap-cutoff-ratio`: (Default 0.25) Percentage of heap space to remove
from containers started by YARN. When a user requests a certain amount of memory for each
TaskManager container (for example 4 GB), we can not pass this amount as the maximum heap
space for the JVM (`-Xmx` argument) because the JVM is also allocating memory outside the
heap. YARN is very strict with killing containers which are using more memory than requested.
Therefore, we remove this fraction of the memory from the requested heap as a safety margin
and add it to the memory used off-heap.
    -- `containerized.heap-cutoff-min`: (Default 600 MB) Minimum amount of memory to cut off
the requested heap size.
    -- `yarn.maximum-failed-containers` (Default: number of requested containers). Maximum
number of containers the system is going to reallocate in case of a failure.
    -- `yarn.application-attempts` (Default: 1). Number of ApplicationMaster restarts. Note
that that the entire Flink cluster will restart and the YARN Client will loose the connection.
Also, the JobManager address will change and you'll need to set the JM host:port manually.
It is recommended to leave this option at 1.
    -- `yarn.heartbeat-delay` (Default: 5 seconds). Time between heartbeats with the ResourceManager.
    -- `yarn.properties-file.location` (Default: temp directory). When a Flink job is submitted
to YARN, the JobManager's host and the number of available processing slots is written into
a properties file, so that the Flink client is able to pick those details up. This configuration
parameter allows changing the default location of that file (for example for environments
sharing a Flink installation between users)
    -- `yarn.containers.vcores` The number of virtual cores (vcores) per YARN container. By
default, the number of `vcores` is set to the number of slots per TaskManager, if set, or
to 1, otherwise.
    -- `containerized.master.env.`*ENV_VAR1=value* Configuration values prefixed with `containerized.master.env.`
will be passed as environment variables to the ApplicationMaster/JobManager process. For example
for passing `LD_LIBRARY_PATH` as an env variable to the ApplicationMaster, set:
    -    `containerized.master.env.LD_LIBRARY_PATH: "/usr/lib/native"`
    -- `containerized.taskmanager.env.` Similar to the configuration prefix about, this prefix
allows setting custom environment variables for the TaskManager processes.
    -- `yarn.container-start-command-template`: Flink uses the following template when starting
on YARN:
    --- End diff --
    looks like you still can. `ConfigConstangs#YARN_CONTAINER_START_COMMAND_TEMPLATE` is read
by the yarn cluster descriptor and BootStrapTools. Looks like we missed porting that option.

> Move remaining sections to generated tables
> -------------------------------------------
>                 Key: FLINK-8475
>                 URL: https://issues.apache.org/jira/browse/FLINK-8475
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Configuration
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>            Priority: Major
>             Fix For: 1.5.0

This message was sent by Atlassian JIRA

View raw message