Return-Path: X-Original-To: apmail-reef-dev-archive@minotaur.apache.org Delivered-To: apmail-reef-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 05DE819700 for ; Mon, 28 Mar 2016 18:31:26 +0000 (UTC) Received: (qmail 57503 invoked by uid 500); 28 Mar 2016 18:31:25 -0000 Delivered-To: apmail-reef-dev-archive@reef.apache.org Received: (qmail 57473 invoked by uid 500); 28 Mar 2016 18:31:25 -0000 Mailing-List: contact dev-help@reef.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@reef.apache.org Delivered-To: mailing list dev@reef.apache.org Received: (qmail 57414 invoked by uid 99); 28 Mar 2016 18:31:25 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Mar 2016 18:31:25 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 88AB02C1F64 for ; Mon, 28 Mar 2016 18:31:25 +0000 (UTC) Date: Mon, 28 Mar 2016 18:31:25 +0000 (UTC) From: "Dhruv Mahajan (JIRA)" To: dev@reef.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (REEF-1224) IMRU Fault Tolerance - Separate Data downloading from Task injection MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/REEF-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214638#comment-15214638 ] Dhruv Mahajan commented on REEF-1224: ------------------------------------- One more question: Suppose I instantiate {{NetworkService}} as part of {{Task}}. Let's assume that task gets completed and exits. Now there would be a lag between when network service is disposed by GC and Evaluator sends message that task is done, right? So even if while disposing we get all these stream closing and read failures message, the task would be deemed as success right? But then when Garbage Collector runs and then these stream errors get thrown, will it throw failed evaluator message? I am little unclear about this. This whole asynchronous Read business is tricky since read are asynchronous. > IMRU Fault Tolerance - Separate Data downloading from Task injection > -------------------------------------------------------------------- > > Key: REEF-1224 > URL: https://issues.apache.org/jira/browse/REEF-1224 > Project: REEF > Issue Type: Improvement > Components: IMRU, REEF.NET > Reporter: Julia > Assignee: Dhruv Mahajan > > Currently in IMRU, data downloading happens during the Task injection. It couples the data and Task object. In Fault tolerant case, we would like to only resubmit a task but use the data that have been downloaded, That requires us to decouple those two portions. For example, data downloading portion can be attached to Context, and we can then resubmit a task on the same context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)