Return-Path: X-Original-To: apmail-flink-issues-archive@minotaur.apache.org Delivered-To: apmail-flink-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 876C518969 for ; Mon, 4 May 2015 13:45:32 +0000 (UTC) Received: (qmail 91730 invoked by uid 500); 4 May 2015 13:45:32 -0000 Delivered-To: apmail-flink-issues-archive@flink.apache.org Received: (qmail 91688 invoked by uid 500); 4 May 2015 13:45:32 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 91679 invoked by uid 99); 4 May 2015 13:45:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 May 2015 13:45:32 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: message received from 54.164.171.186 which is an MX secondary for issues@flink.apache.org) Received: from [54.164.171.186] (HELO mx1-us-east.apache.org) (54.164.171.186) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 May 2015 13:45:27 +0000 Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id 7ED3A47362 for ; Mon, 4 May 2015 13:45:07 +0000 (UTC) Received: (qmail 90688 invoked by uid 99); 4 May 2015 13:45:07 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 May 2015 13:45:07 +0000 Date: Mon, 4 May 2015 13:45:07 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@flink.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (FLINK-1672) Refactor task registration/unregistration MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/FLINK-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526639#comment-14526639 ] ASF GitHub Bot commented on FLINK-1672: --------------------------------------- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/646#issuecomment-98707668 Subsumed by #648 > Refactor task registration/unregistration > ----------------------------------------- > > Key: FLINK-1672 > URL: https://issues.apache.org/jira/browse/FLINK-1672 > Project: Flink > Issue Type: Improvement > Components: Distributed Runtime > Reporter: Ufuk Celebi > > h4. Current control flow for task registrations > # JM submits a TaskDeploymentDescriptor to a TM > ## TM registers the required JAR files with the LibraryCacheManager and returns the user code class loader > ## TM creates a Task instance and registers the task in the runningTasks map > ## TM creates a TaskInputSplitProvider > ## TM creates a RuntimeEnvironment and sets it as the environment for the task > ## TM registers the task with the network environment > ## TM sends async msg to profiler to monitor tasks > ## TM creates temporary files in file cache > ## TM tries to start the task > If any operation >= 1.2 fails: > * TM calls task.failExternally() > * TM removes temporary files from file cache > * TM unregisters the task from the network environment > * TM sends async msg to profiler to unmonitor tasks > * TM calls unregisterMemoryManager on task > If 1.1 fails, only unregister from LibraryCacheManager. > h4. RuntimeEnvironment, Task, TaskManager separation > The RuntimeEnvironment has references to certain components of the task manager like memory manager, which are accecssed from the task. Furthermore it implements Runnable, and creates the executing task Thread. The Task instance essentially wraps the RuntimeEnvironment and allows asynchronous state management of the task (RUNNING, FINISHED, etc.). > The way that the state updates affect the task is not that obvious: state changes trigger messages to the TM, which for final states further trigger a msg to unregister the task. The way that tasks are unregistered again depends on the state of the task. > ---- > I would propose to refactor this to make the way the state handling/registration/unregistration is handled is more transparent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)