reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia Wang (QIUHE)" <Qiuhe.W...@microsoft.com>
Subject RE: ICompletedTask semantics
Date Fri, 08 Apr 2016 18:39:59 GMT
I agree. If task is completed, we should try to dispose task. But if dispose fails, we should
ignore it. 

After driver receives task completed event, driver doesn't acknowledge evaluator, right? It
is evaluator, that finally dispose the task. The sequence 1. Set task status to Complete 2.
dispose task (ignore error if any) 3. Then send event back to driver seems more reasonable
to me if the change is not hard. 

Julia

-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com] 
Sent: Friday, April 8, 2016 11:29 AM
To: dev@reef.apache.org
Subject: Re: ICompletedTask semantics

I feel if dispose fails, Task should still be termed as completed and failure should be ignored.
Can we think of scenarios in which this logic is bad?

Dhruv

On Fri, Apr 8, 2016 at 11:18 AM, Andrew Chung <afchung90@gmail.com> wrote:

> Hi Dhruv,
>
> Interesting question. There might actually be a race condition here 
> (somewhat, since I'm not sure if we count the `Dispose` call as part 
> of the Task), given that `Dispose` is actually called *after* setting 
> the Task status to `Completed`, and the Task thread locks on a 
> separate object than the Heartbeat thread. Only the Task status set 
> operation is atomic. Thus, if the Heartbeat thread gets the Task 
> status and sends the message to the Driver prior to the Dispose is 
> called, it is possible that Task.Dispose is not yet called before the 
> Driver receives the Task completion message.
>
> Given that we haven't defined a particular order for this sequence of 
> events, perhaps we should change the behavior to what Dhruv expects?
>
> Another thing to consider is whether we count `Dispose` as part of the 
> Task. e.g. If Dispose fails, do we count the Task as failed (Somewhat 
> dangerous, since many users don't implement Dispose on their Tasks)?
> Do we only count the Task as Completed after the Dispose is done? What 
> do you think?
>
> Thanks,
> Andrew
>
> On Fri, Apr 8, 2016 at 10:26 AM, Julia Wang (QIUHE) 
> <Qiuhe.Wang@microsoft.com> wrote:
> > It has been disposed I believe.
> >
> > -----Original Message-----
> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> > Sent: Friday, April 8, 2016 10:07 AM
> > To: dev@reef.apache.org
> > Subject: ICompletedTask semantics
> >
> > Hi
> >
> > I have a question regarding the semantics of ICompletedTask. Does 
> > this
> mean that Dispose on the Task has already been called or it will be 
> called after driver acks. this message? I feel it is former but just 
> wanted to verify.
> >
> > Dhruv
>
Mime
View raw message