flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (FLINK-1672) Refactor task registration/unregistration
Date Mon, 11 May 2015 20:26:59 GMT

     [ https://issues.apache.org/jira/browse/FLINK-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Stephan Ewen resolved FLINK-1672.
       Resolution: Fixed
    Fix Version/s: 0.9
         Assignee: Stephan Ewen

Implemented in 8e61301452218e6d279b013beb7bbd02a7c2e3f9

> Refactor task registration/unregistration
> -----------------------------------------
>                 Key: FLINK-1672
>                 URL: https://issues.apache.org/jira/browse/FLINK-1672
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Runtime
>            Reporter: Ufuk Celebi
>            Assignee: Stephan Ewen
>             Fix For: 0.9
> h4. Current control flow for task registrations
> # JM submits a TaskDeploymentDescriptor to a TM
> ## TM registers the required JAR files with the LibraryCacheManager and returns the user
code class loader
> ## TM creates a Task instance and registers the task in the runningTasks map
> ## TM creates a TaskInputSplitProvider
> ## TM creates a RuntimeEnvironment and sets it as the environment for the task
> ## TM registers the task with the network environment
> ## TM sends async msg to profiler to monitor tasks
> ## TM creates temporary files in file cache
> ## TM tries to start the task
> If any operation >= 1.2 fails:
> * TM calls task.failExternally()
> * TM removes temporary files from file cache
> * TM unregisters the task from the network environment
> * TM sends async msg to profiler to unmonitor tasks
> * TM calls unregisterMemoryManager on task
> If 1.1 fails, only unregister from LibraryCacheManager.
> h4. RuntimeEnvironment, Task, TaskManager separation
> The RuntimeEnvironment has references to certain components of the task manager like
memory manager, which are accecssed from the task. Furthermore it implements Runnable, and
creates the executing task Thread. The Task instance essentially wraps the RuntimeEnvironment
and allows asynchronous state management of the task (RUNNING, FINISHED, etc.).
> The way that the state updates affect the task is not that obvious: state changes trigger
messages to the TM, which for final states further trigger a msg to unregister the task. The
way that tasks are unregistered again depends on the state of the task.
> ----
> I would propose to refactor this to make the way the state handling/registration/unregistration
is handled is more transparent.

This message was sent by Atlassian JIRA

View raw message