Mailing-List: contact issues-help@mesos.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mesos.apache.org
Date: Tue, 18 Nov 2014 00:26:33 +0000 (UTC)
From: "Zach Carlson (JIRA)" <jira@apache.org>
To: issues@mesos.apache.org
Message-ID: <JIRA.12755952.1416266942000.520188.1416270393660@Atlassian.JIRA>
In-Reply-To: <JIRA.12755952.1416266942000@Atlassian.JIRA>
References: <JIRA.12755952.1416266942000@Atlassian.JIRA>
 <JIRA.12755952.1416266942183@arcas>
Subject: [jira] [Updated] (MESOS-2122) MesosSchedulerDriver stop causes
 resource offer exhaustion
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/MESOS-2122?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zach Carlson updated MESOS-2122:
--------------------------------
    Affects Version/s:     (was: 0.21.0)

> MesosSchedulerDriver stop causes resource offer exhaustion
> ----------------------------------------------------------
>
>                 Key: MESOS-2122
>                 URL: https://issues.apache.org/jira/browse/MESOS-2122
>             Project: Mesos
>          Issue Type: Bug
>    Affects Versions: 0.20.0, 0.20.1
>         Environment: x86_64 Debian Wheezy (w/ mesosphere repos, packages)
>            Reporter: Zach Carlson
>         Attachments: mesos_2122.py
>
>
> For additional consideration, see https://github.com/airbnb/chronos/issue=
s/290 and https://github.com/mesosphere/marathon/issues/787
> When the SchedulerProcess managed by the MesosSchedulerDriver detects a m=
aster, it performs a link() to the master. Libprocess proceeds to establish=
 the link. Once the scheduler has performed all the work necessary, it may =
call MesosSchedulerDriver.stop(failover =3D true).=20
> This is where things go awry: at this point, the SchedulerProcess schedul=
es a termination event for itself. When libprocess's schedule thread rolls =
through, it performs a cleanup() of the SchedulerProcess, as expected. Part=
 of the cleanup() is calling SocketManager::exited() on the SchedulerProces=
s. The problem with this is that SocketManager::exited() cleans up the link=
s from the link map, but does not actually close the sockets. Now, since Me=
sosSchedulerDriver::stop() was called with failover =3D true, no Deregister=
Framework message was sent, so the Mesos master believes that the connectio=
n (which is still active) is still valid with a registered framework listen=
ing for events. It sends resourceOffers to the 'valid' framework... and sin=
ce there's nothing actually listening for events, no response is sent, no o=
ffers are accepted or declined, and Mesos will grind to a halt (*until vers=
ion 0.21.0, which will (according to release notes) rescind un-responded of=
fers after a configurable timeout) -- no further offers made to any framewo=
rk, and when all current framework work has completed, no further work will=
 be performed due to the offers being wasted.=20


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)