reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Weimer <mar...@weimo.de>
Subject Re: [REEF-1778] The IIMRUResultHandler Dispose() method isn't allowed to complete before the job finishes.
Date Mon, 17 Apr 2017 16:34:16 GMT
Thanks so much for reporting this in detail! This looks like a bug in
REEF, not even IMRU to me. I have posted some thoughts on the JIRA as
to what the root cause is and how to address it.

Markus

On Fri, Apr 14, 2017 at 6:07 PM, Rogan Carr <rogan.carr@gmail.com> wrote:
> Hi All,
>
> I have run into an interesting issue in the IMRU API where there seems to
> be a race condition at the end of the job between the Dispose() method in
> the IIMRUResultHandler (and therefore in the UpdateTaskHost) and whatever
> signals that the job should end. I have created [REEF-1778] "The
> IIMRUResultHandler Dispose() method isn't allowed to complete before the
> job finishes" to document this issue, including logs that clearly show that
> Dispose() isn't done when the job ends.
>
> My hunch is that the driver is calling Dispose() on the UpdateTaskHost()
> and then telling YARN that it is finished, and then YARN kills the
> container.
>
> Does anybody have any thoughts on what the issue might be?
>
> Thanks for your help!
>
> Best,
> Rogan
>
> [1] https://issues.apache.org/jira/browse/REEF-1778

Mime
View raw message