flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaogang Shi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4408) Submit Job and setup ExecutionGraph
Date Wed, 24 Aug 2016 01:54:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434052#comment-15434052

Xiaogang Shi commented on FLINK-4408:

I think it's okay to pull the logics about leadership out of the JobManager and let the component
holding the JobManager to take care of it.

But I think the JobManager still needs to worry about the switches.  The JobManager still
need the function to submit/execute the job when it is started by the component holding it
(to be specific, the `submitJob` function in old JobManager).  The only difference made by
pulling the logics out is that the function will be called by the component holding the JobManager
but not the JobManager itself. 

If I understand your comment correctly, i think we should revise the implementation in Flink-4400
to let JobManager not care about leadership. But the JobManager still needs to implement the
methods to start and cancel the execution, which are implemented in this JIRA.

> Submit Job and setup ExecutionGraph
> -----------------------------------
>                 Key: FLINK-4408
>                 URL: https://issues.apache.org/jira/browse/FLINK-4408
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Cluster Management
>            Reporter: Xiaogang Shi
>            Assignee: Xiaogang Shi
> Once granted the leadership, JM will start to execute the job.
> Most code remains the same except that 
> (1) In old implementation where JM manages the execution of multiple jobs, JM has to
load all submitted JobGraphs from SubmittedJobGraphStore and recover them. Now that the components
creating JM will be responsible for the recovery of JobGraphs, JM will be created with submitted/recovered
JobGraph, without the need to load the JobGraph.
> (2) JM should not rely on Akka to listen on the updates of JobStatus and Execution.

This message was sent by Atlassian JIRA

View raw message