flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Wang (Jira)" <j...@apache.org>
Subject [jira] [Created] (FLINK-19543) Implement RunningJobsRegistry, JobGraphStore based on Kubernetes API
Date Fri, 09 Oct 2020 03:08:00 GMT
Yang Wang created FLINK-19543:
---------------------------------

             Summary: Implement RunningJobsRegistry, JobGraphStore based on Kubernetes API
                 Key: FLINK-19543
                 URL: https://issues.apache.org/jira/browse/FLINK-19543
             Project: Flink
          Issue Type: Sub-task
          Components: Deployment / Kubernetes, Runtime / Coordination
            Reporter: Yang Wang
             Fix For: 1.12.0


* *_RunningJobsRegistry_*
 * Registry for the running jobs. All the jobs in the registry will be recovered when JobManager
failover. 


 * *_JobGraphStore_*
 * JobGraph instances for running JobManagers. Note that only the meta information(aka location
reference, DFS path) will be stored in the Zookeeper/ConfigMap. The real data is stored on
the DFS.

Each component(Dispatcher, ResourceManager, JobManager, RestEndpoint) will have a dedicated
ConfigMap. All the HA information relevant for a specific component will be stored in a
single ConfigMap. So the Dispatcher's ConfigMap would then contain the current leader, the
running jobs and the pointers to the persisted JobGraphs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message