mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Park (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MESOS-7851) Master stores old resource format in the registry
Date Mon, 23 Oct 2017 06:24:00 GMT

    [ https://issues.apache.org/jira/browse/MESOS-7851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16142405#comment-16142405
] 

Michael Park edited comment on MESOS-7851 at 10/23/17 6:23 AM:
---------------------------------------------------------------

Just writing down what should be done here.

The master has 2 things that contain resources that go into the registry: {{SlaveInfo}}, and
{{QuotaInfo}}.
In order to support master downgrades (e.g., 1.4.0 => 1.3.1), we must store the resources
in the "pre-reservation-refinement" format. This happens for {{SlaveInfo}} today (albeit incidentally),
but not
for {{QuotaInfo}}.

Resources inside {{QuotaInfo}} should probably be downgraded for the {{Registry}} and upgraded
on their way out. However, with the current requirement that {{QuotaInfo}} can only hold unreserved
resources, we don't need to do anything for this. (tested manually by setting a quota with
1.4.0 master, downgrading to 1.3.1 and hitting the quota endpoint).

Changes that should to be made:
  - {{Master::_recover}} should upgrade the resources inside {{SlaveInfo}} before inserting
it into the {{slaves.recovered}} map. {{authorizeResources}} can be fixed after this. (tech
debt)


was (Author: mcypark):
Just writing down what should be done here.

The master has 2 things that contain resources that go into the registry: {{SlaveInfo}}, and
{{QuotaInfo}}.
In order to support master downgrades (e.g., 1.4.0 => 1.3.1), we must store the resources
in the "pre-reservation-refinement" format. This happens for {{SlaveInfo}} today (albeit incidentally),
but not
for {{QuotaInfo}}.

Resources inside {{QuotaInfo}} should probably be downgraded for the {{Registry}} and upgraded
on their way out. However, with the current requirement that {{QuotaInfo}} can only hold unreserved
resources, we don't need to do anything for this. (tested manually by setting a quota with
1.4.0 master, downgrading to 1.3.1 and hitting the quota endpoint).

Changes that should to be made:
  - {{Master::_recover}} should upgrade the resources inside {{SlaveInfo}} before inserting
it into the {{slaves.recovered}} map. {{authorizeResources}} can be fixed after this. (tech
debt)
  - The master should upgrade the resources inside {{SlaveInfo}} earlier in the {{(re)registerSlave}}
handlers, and use the downgraded version just for the {{Registry}}. (clean-up)

> Master stores old resource format in the registry
> -------------------------------------------------
>
>                 Key: MESOS-7851
>                 URL: https://issues.apache.org/jira/browse/MESOS-7851
>             Project: Mesos
>          Issue Type: Bug
>          Components: master
>            Reporter: Greg Mann
>            Assignee: Michael Park
>              Labels: master, mesosphere, reservation
>
> We intend for the master to store all internal resource representations in the new, post-reservation-refinement
format. However, [when persisting registered agents to the registrar|https://github.com/apache/mesos/blob/498a000ac1bb8f51dc871f22aea265424a407a17/src/master/master.cpp#L5861-L5876],
the master does not convert the resources; agents provide resources in the pre-reservation-refinement
format, and these resources are stored as-is. This means that after recovery, any agents in
the master's {{slaves.recovered}} map will have {{SlaveInfo.resources}} in the pre-reservation-refinement
format.
> We should update the master to convert these resources before persisting them to the
registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message