mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-7926) Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.
Date Fri, 01 Sep 2017 02:54:00 GMT

    [ https://issues.apache.org/jira/browse/MESOS-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149953#comment-16149953
] 

Jie Yu commented on MESOS-7926:
-------------------------------

commit 2e96672b55a17c49f5228d8629f799fca4df0fdb
Author: Jie Yu <yujie.jay@gmail.com>
Date:   Wed Aug 30 17:28:05 2017 -0700

    Used 'undiscardable' to protect a future in MesosContainerizer.

    This patch addressed MESOS-7926 by protecting 'termination.future()'
    with 'undiscardable'. This prevented the upstream discard event from
    being propagated into 'termination.future()' which gets reused for
    other calls.



> Abnormal termination of default executor can cause MesosContainerizer::destroy to fail.

> ----------------------------------------------------------------------------------------
>
>                 Key: MESOS-7926
>                 URL: https://issues.apache.org/jira/browse/MESOS-7926
>             Project: Mesos
>          Issue Type: Bug
>    Affects Versions: 1.2.2, 1.3.1, 1.4.0
>            Reporter: Jie Yu
>            Priority: Critical
>
> This is the sequence of events:
> 1) Default executor launches a nested container
> 2) Default executor invokes agent API WAIT_NESTED_CONTAINER
> 3) Default executor is killed
> 4) The connection to the agent for WAIT_NESTED_CONTAINER breaks
> 5) libprocess discard the future, which propagates to the code [here|https://github.com/apache/mesos/blob/1.4.0-rc3/src/slave/containerizer/mesos/containerizer.cpp#L1955-L1956].
> 6) `termination.future()` has the discard flag being set to true (i.e., hasDiscard()
== true).
> 7) Default executor termination triggers container destroy for the nested container
> 8) When the destroy of the nested container is done, the control will reach [here|https://github.com/apache/mesos/blob/1.4.0-rc3/src/slave/containerizer/mesos/containerizer.cpp#L2176-L2177].
> 9) In the [thenf|https://github.com/apache/mesos/blob/1.4.0-rc3/3rdparty/libprocess/include/process/future.hpp#L1299-L1301]
handler for 'termination.future()', since `termination.future()` has discard flag set (hasDiscard()
== true), we'll call promise->discard(), which cause the returned future to be in DISCARDED
state.
> 10) The top level container destroy will fail because nested container destroy failed
> 11) This cause all isolator cleanup for the top level container not being called (e.g.,
CNI detach).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message