mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Park (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-7735) The master crashes when state endpoint is hit during a task authorization.
Date Wed, 05 Jul 2017 10:57:00 GMT

    [ https://issues.apache.org/jira/browse/MESOS-7735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074575#comment-16074575
] 

Michael Park commented on MESOS-7735:
-------------------------------------

{noformat}
commit e097f21124d16c80544af7aabc83bab5fb712453
Author: Michael Park <mpark@apache.org>
Date:   Thu Jun 29 23:49:10 2017 -0700

    Performed validation/normalization of `Resource`s before authorization.

    Review: https://reviews.apache.org/r/60564
{noformat}
{noformat}
commit 63c8b1d63be363e96d7c2672d9d65b9d67ca48ed
Author: Michael Park <mpark@apache.org>
Date:   Thu Jun 29 23:30:16 2017 -0700

    Updated `validateAndNormalizeResources` to operate on `Operation`s.

    Review: https://reviews.apache.org/r/60563
{noformat}
{noformat}
commit 710b72179938cac2100c90277ce7ced5c8ca3401
Author: Michael Park <mpark@apache.org>
Date:   Thu Jun 29 20:53:59 2017 -0700

    Updated `accept` to perform operation adjustment in one place.

    It used to be that the minor adjustments that were made to operations
    were done in various places across `accept` and `_accept`.

    The "executor-injection" for LAUNCH_GROUP was at the beginning of
    `accept`, "allocation-info-injection" for MULTI_ROLE was after offer
    validation, and "health-check-injection" for LAUNCH was in `_accept`.

    The `Master::accept` function is now broken down into distinct
    "metrics accounting", "offer validation", "operation-adjustments", and
    "authorization" stages.

    Review: https://reviews.apache.org/r/60562
{noformat}

> The master crashes when state endpoint is hit during a task authorization.
> --------------------------------------------------------------------------
>
>                 Key: MESOS-7735
>                 URL: https://issues.apache.org/jira/browse/MESOS-7735
>             Project: Mesos
>          Issue Type: Bug
>          Components: master
>            Reporter: Michael Park
>            Assignee: Michael Park
>            Priority: Blocker
>
> With reservation refinement, the construction of {{Resources}} requires {{Resource}}
objects to have been validated and converted to the "post-reservation-refinement" format.
Generally speaking, validation and conversion are the first steps we take with given {{Resource}}
objects prior to proceeding. In the master currently, we perform authorization first with
not-yet-validated, not-yet-converted {{Resource}} objects. During the authorization phase,
we add tasks with not-yet-validated, not-yet-converted resources into {{framework->pendingTasks}}
as well as {{slave->pendingTasks}}. (https://github.com/apache/mesos/blob/master/src/master/master.cpp#L3974-L3999).
If one hits the state endpoint on the master during this time, we get to https://github.com/apache/mesos/blob/master/src/master/http.cpp#L278
which tries to construct a {{Resources}} with {{taskInfo.resources()}} which is not-yet-validated
nor converted.
> I think the correct fix here is to perform validation / conversion prior to authorization.
The authorization code currently is written to carefully inspect fields in both "pre-reservation-refinement"
and "post-reservation-refinement" formats. By performing validation / conversion first, the
authorization code would be simplified, and we're also much less likely to make mistakes such
as this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message