flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4913) Per-job Yarn clusters: include user jar in system class loader
Date Tue, 25 Oct 2016 16:14:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605731#comment-15605731
] 

ASF GitHub Bot commented on FLINK-4913:
---------------------------------------

GitHub user mxm opened a pull request:

    https://github.com/apache/flink/pull/2692

    [FLINK-4913][yarn] include user jar in system class loader

    When deploying a Yarn cluster for a single job, this change
    pre-configures the cluster to include the user jar(s) on all nodes.
    This eliminates the need to upload jar files through the
    BlobClient. More importantly, it loads the user classes only once and
    not on every instantiation of a Task. This also reduces the JobManager
    class loading upon recovery of a failed job.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mxm/flink FLINK-4913

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2692.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2692
    
----
commit bfe0f1a1b5cfee200ff76e4b988fe43613c8d0c0
Author: Maximilian Michels <mxm@apache.org>
Date:   2016-10-25T14:16:52Z

    [FLINK-4913][yarn] include user jar in system class loader
    
    When deploying a Yarn cluster for a single job, this change
    pre-configures the cluster to include the user jar(s) on all nodes.
    This eliminates the need to upload jar files through the
    BlobClient. More importantly, it loads the user classes only once and
    not on every instantiation of a Task. This also reduces the JobManager
    class loading upon recovery of a failed job.

----


> Per-job Yarn clusters: include user jar in system class loader 
> ---------------------------------------------------------------
>
>                 Key: FLINK-4913
>                 URL: https://issues.apache.org/jira/browse/FLINK-4913
>             Project: Flink
>          Issue Type: Improvement
>          Components: Client, YARN Client
>            Reporter: Maximilian Michels
>            Assignee: Maximilian Michels
>             Fix For: 1.2.0, 1.1.4
>
>
> Including the jar directly in the system classloader avoids loading it for every instantiation
of the ExecutionGraph and every Task execution. Note, this is only possible for per-job clusters
(i.e. Yarn/Mesos).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message