flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Wang (Jira)" <j...@apache.org>
Subject [jira] [Created] (FLINK-19543) Implement RunningJobsRegistry, JobGraphStore based on Kubernetes API
Date Fri, 09 Oct 2020 03:08:00 GMT
Yang Wang created FLINK-19543:

             Summary: Implement RunningJobsRegistry, JobGraphStore based on Kubernetes API
                 Key: FLINK-19543
                 URL: https://issues.apache.org/jira/browse/FLINK-19543
             Project: Flink
          Issue Type: Sub-task
          Components: Deployment / Kubernetes, Runtime / Coordination
            Reporter: Yang Wang
             Fix For: 1.12.0

* *_RunningJobsRegistry_*
 * Registry for the running jobs. All the jobs in the registry will be recovered when JobManager

 * *_JobGraphStore_*
 * JobGraph instances for running JobManagers. Note that only the meta information(aka location
reference, DFS path) will be stored in the Zookeeper/ConfigMap. The real data is stored on
the DFS.

Each component(Dispatcher, ResourceManager, JobManager, RestEndpoint) will have a dedicated
ConfigMap. All the HA information relevant for a specific component will be stored in a
single ConfigMap. So the Dispatcher's ConfigMap would then contain the current leader, the
running jobs and the pointers to the persisted JobGraphs.

This message was sent by Atlassian Jira

View raw message