mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Klaus Ma <klaus1982...@gmail.com>
Subject Re: Dynamic vs. implicit roles
Date Tue, 01 Dec 2015 06:01:56 GMT
@Neil, just want to confirm about ACL, do you mean we will load role info
from 3rd part application, e.g. LDAP?

And as I mentioned in both design doc, why not build a RoleManager as
plugin for them? Both features are required following operator:
1. check: check whether role is available
2. create: create role in Master
3. update: update role info
4. destroy: delete the role
5. persist:
6. query: query from role manager.
   master/allocator need role info during the operation

For the life cycle of role, "update" should be period & manual (API/HTT);
others are trigger by API/HTTP.


----
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform Symphony/DCOS Development & Support, STG, IBM GCG
+86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me

On Tue, Dec 1, 2015 at 1:13 PM, James Peach <jorgar@gmail.com> wrote:

>
> > On Nov 30, 2015, at 6:53 PM, YongQiao Wang <jamesyongqiao@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > Currently, there are two proposals on how to improve role management in
> > Mesos:
> >
> > (a) Dynamic roles (MESOS-3177): roles are stored in the registry
> > and queried/added/deleted/removed via HTTP endpoints. I posted a design
> doc
> > here:
> >
> https://docs.google.com/document/d/1OIgceqpsjV3-_LGF83IMAFnrh1Ea3Zc16w9kWWPpUj4/edit#
> >
> > (b) Implicit Roles (MESOS-3988): any role will be allowed, subject to the
> > ACL/authorization system. In a sense, "all roles" exist, so there is no
> > need to store the set of legal roles or provide endpoints to modify them.
> > Neil also has posted a design doc here:
> >
> https://docs.google.com/document/d/1SCFfrBd4edSY3bVCMrNJYMxIVllD0bHJuGmgG-4vCXA/edit?usp=sharing
>
> Implicit roles seem to behave more like groups in an OS where the role
> string is more gid_t which can be consumed directly by the kernel. I think
> that this is moving in the right direction since you shouldn't need to
> exhaustively know all the roles as long as you can attribute resources to
> them correctly. I can imagine that in future you could have an external
> identity service that signs a binding between frameworks and roles that
> Mesos can verify.
>
> > We have discussed in the previous session, append the discussion history
> as
> > below. Let's have a further discussion to choose a better solution
> between
> > them, any comments and feedbacks would be very welcome!
> >
> > ----- Original message -----
> > From: Yong Qiao Wang/China/IBM
> > To: neil@mesosphere.io
> > Cc: adam@mesosphere.io, benh@mesosphere.io, Qian AZ
> Zhang/China/IBM@IBMCN,
> > yongfeng@ca.ibm.com, jamesyongqiao@gmail.com
> > Subject: Re: Dynamic vs. implicit roles
> > Date: Tue, Dec 1, 2015 10:27 AM
> >
> > Thanks Neil.
> >
> >> With implicit roles, that would involve:
> >>
> >> 1. Choosing a role name
> >> 2. Configuring weights, ACLs, and quotas for the role.
> >> 3. Configuring applications/frameworks to register using that role.
> >
> > [Yong Qiao] If applications/frameworks do not follow your rules, and
> > register with another role, then how to prevent? and do we will still
> > create this undesirable role in Mesos? Maybe we can only relay on ACLs to
> > avoid this, but according to my understanding, ACLs is not required in
> > Mesos. In addition, I am not sure whether it is make sence to use ACLs
> for
> > role validation.
> >
> > Regards!
> > *Yong Qiao Wang*
> >
> > Re: Dynamic vs. implicit roles
> > 6:55 AM
> > Neil Conway to me (cc), Yong Feng, Qian AZ Zhang, benh@mesosphere.io,
> Adam
> > Bordelon
> > Show more
> > Hi Yong,
> >
> > Thanks for your feedback.
> >
> > On Mon, Nov 30, 2015 at 2:36 PM, Yong Feng <yongfeng@ca.ibm.com> wrote:
> >
> > 2. MESOS-3988 treats it as part of framework API. I assume that the new
> > created implicit role only has a kind of default resource plan like
> > weight/quota.
> >
> >
> > This is not the case: if you configure a weight/quota/ACL for a role,
> that
> > configuration will be used whenever a framework tries to register using
> > that role. So you can configure a non-default weight/quota/etc. for an
> > implicit role just as you would with a statically or dynamically
> configured
> > role.
> >
> > We still rely on management API such as quota management to further
> > configure the resource plan.
> >
> > For the use case that in a company/organization, all resources are
> > allocated according to pre-defined budget plan. We will need admin to
> > create role, configure resource plan before launching application.
> > Mesos-3988 does not help as the resource plan should be configured before
> > application is running.
> >
> >
> > I would phrase it as: in both designs, the admin will configure a
> "resource
> > plan". With dynamic roles, that would involve:
> >
> > 1. Choosing a role name
> > 2. Creating a role with that name
> > 3. Configuring weights, ACLs, and quotas for the role.
> > 4. Configuring applications/frameworks to register using that role.
> >
> > With implicit roles, that would involve:
> >
> > 1. Choosing a role name
> > 2. Configuring weights, ACLs, and quotas for the role.
> > 3. Configuring applications/frameworks to register using that role.
> >
> > i.e., implicit roles are equivalent to dynamic roles, but slightly
> simpler.
> > If you see situations in which dynamic roles would allow you to do
> > something that implicit roles would not, please let me know -- I'm not
> > aware of any situations myself.
> >
> > For the use case that in a cloud environment, user would like to launch
> app
> > for a certain service level agreement. It does not make much sense to
> > create a role for the application only and in advance. We could simply
> > create a role for the "service level agreement", and then ask application
> > register with the role. Regarding to allocating resources among
> frameworks
> > within the same service level agreement, we already have object of
> > "framework" which is used as an entity when Mesos allocate resources.
> >
> >
> > Sorry, I didn't quite follow what you mean here.
> >
> > Thanks,
> > Neil
> >
> > To: Adam Bordelon <adam@mesosphere.io>
> > From: Yong Feng/Markham/IBM
> > Date: 12/01/2015 06:36AM
> > Cc: Benjamin Hindman <benh@mesosphere.io>, Neil Conway <
> neil@mesosphere.io>,
> > Yong Qiao Wang <yqwyq@cn.ibm.com>, Qian AZ Zhang <zhangqxa@cn.ibm.com>
> > Subject: Re: Dynamic vs. implicit roles
> >
> > We'd better to move it into dev@list. Just try to show my two cents in
> this
> > tread
> >
> > I see the main difference between the two proposals is that
> >
> > 1. MESOS-3177 introduces management API for role life cycle management
> and
> > in future for how to plan resource among tenants. We usually call
> "planning
> > resource among tenants" as resource plan. The quota management actually
> > also belongs to it.
> > 2. MESOS-3988 treats it as part of framework API. I assume that the new
> > created implicit role only has a kind of default resource plan like
> > weight/quota. We still rely on management API such as quota management to
> > further configure the resource plan.
> >
> > For the use case that in a company/organization, all resources are
> > allocated according to pre-defined budget plan. We will need admin to
> > create role, configure resource plan before launching application.
> > Mesos-3988 does not help as the resource plan should be configured before
> > application is running.
> >
> > For the use case that in a cloud environment, user would like to launch
> app
> > for a certain service level agreement. It does not make much sense to
> > create a role for the application only and in advance. We could simply
> > create a role for the "service level agreement", and then ask application
> > register with the role. Regarding to allocating resources among
> frameworks
> > within the same service level agreement, we already have object of
> > "framework" which is used as an entity when Mesos allocate resources.
> >
> > So basically I did not see a strong use case Mesos-3988 could  resolve
> > while MESOS-3177 does not or need extra efforts. However I do see the use
> > cases Mesos-3988 cannot resolve.
> >
> > Thanks,
> >
> > Yong
> >
> > [image: Inactive hide details for Adam Bordelon ---11/30/2015 03:26:55
> > AM---- In the implicit roles model, new roles are "created" when]Adam
> > Bordelon ---11/30/2015 03:26:55 AM---- In the implicit roles model, new
> > roles are "created" when a framework successfully registers under
> >
> > From: Adam Bordelon <adam@mesosphere.io>
> > To: Yong Qiao Wang <yqwyq@cn.ibm.com>
> > Cc: Neil Conway <neil@mesosphere.io>, Benjamin Hindman <
> benh@mesosphere.io>,
> > Qian AZ Zhang <zhangqxa@cn.ibm.com>, Yong Feng/Markham/IBM@IBMCA
> > Date: 11/30/2015 03:26 AM
> > Subject: Re: Dynamic vs. implicit roles
> > ------------------------------
> >
> >
> >
> > - In the implicit roles model, new roles are "created" when a framework
> > successfully registers under that role. Other actions like creating a
> > reservation/volume or setting a weight/quota implicitly "create" or
> "name"
> > a role, but it isn't active until there's a framework registered.
> > - A role exists as long as it still has any registered frameworks that
> > haven't timed out yet. Even then, it isn't really active in the allocator
> > unless at least one of its frameworks is active, even if there are
> > reservations/volumes/quota/weights associated.
> > - Implicit roles don't need to be persisted, if any role is allowed. If
> > there are ACLs restricting the set of allowed roles, then those ACLs will
> > have to be stored statically in the master's --acls flag, or eventually
> in
> > the replicated log when we have dynamic ACLs.
> > - Typos are unfortunate, but we have to trust that the operator can be
> > consistent when configuring ACLs, weights, quota, etc. If we need an
> > explicit role whitelist, then we can use ACLs to express that only
> certain
> > roles are allowed.
> >
> > Let's take future conversations onto the dev@ list so we can get others
> > involved.
> >
> > On Sun, Nov 29, 2015 at 9:59 PM, Yong Qiao Wang <*yqwyq@cn.ibm.com*
> > <yqwyq@cn.ibm.com>> wrote:
> >
> >   In addition, Dynamic roles/weights(MESOS-3177) proposes a simplified
> and
> >   centralized management(Creating/Removing/Updating/Persisting) for role
> life
> >   cycle.  If we propose to use "Implicit Roles(MESOS-3988)" to replace
> >   "Dynamic Roles(MESOS-3177)", then I want to know how we will cover the
> same
> >   functions in Implicit Roles, for example:
> >
> >   - When create a role in Mesos?
> >   - When delete a role in Mesos?
> >   - How to persist roles in replicated log?
> >   - How to avoid the typos? For example, role typos when framework
> >   register, when configure ACLs, when configure weight, etc.
> >
> >   Neil, could you also help to clarify above concerns in your design doc?
> >   It is important to help us to make a right decision between Dynamic
> Roles
> >   and Implicit Roles.
> >
> >   Thanks!
> >   *Yong Qiao Wang*
> >
> >
> >      ----- Original message -----
> >      From: Yong Qiao Wang/China/IBM
> >      To: *neil@mesosphere.io* <neil@mesosphere.io>
> >      Cc: *adam@mesosphere.io* <adam@mesosphere.io>, *benh@mesosphere.io*
> >      <benh@mesosphere.io>
> >      Subject: Re: Dynamic vs. implicit roles
> >      Date: Sat, Nov 28, 2015 10:07 PM
> >
> >      Hi Neil,
> >
> >      Thanks a million for your proposals, sorry for being so unresponsive
> >      lately, I have been taking vacation.
> >
> >      I have read the design doc of implicit roles, one main commnet as
> >      below:
> >
> >      Per my understanding, in Mesos, the role which is used to determine
> >      what resources frameworks can use, and the total number of roles
> affects
> >      each role's fair share of the Mesos cluster, but in the proposal of
> >      implicit roles, framework can register with all possible role, so
> how to
> >      guarantee this?
> >
> >      Regards!
> >      *Yong Qiao Wang*
> >
> >
> >         ----- Original message -----
> >         From: Neil Conway <*neil@mesosphere.io* <neil@mesosphere.io>>
> >         To: Yong Qiao Wang/China/IBM@IBMCN
> >         Cc: Benjamin Hindman <*benh@mesosphere.io* <benh@mesosphere.io
> >>,
> >         Adam Bordelon <*adam@mesosphere.io* <adam@mesosphere.io>>
> >         Subject: Dynamic vs. implicit roles
> >         Date: Thu, Nov 26, 2015 7:36 AM
> >
> >         Hi Yong,
> >
> >         We've both been looking at how to improve role management in
> >         Mesos, so
> >         I wanted to get in touch about the best way to move forward here.
> >
> >         As you know, there are two proposals:
> >
> >         (a) Dynamic roles (MESOS-3177): roles are stored in the registry
> >         and
> >         added/deleted/removed via HTTP endpoints
> >
> >         (b) Implicit roles (MESOS-3988): any role will be allowed,
> subject
> >         to
> >         the ACL/authorization system. In a sense, "all roles" exist, so
> >         there
> >         is no need to store the set of legal roles or provide endpoints
> to
> >         modify them. For more information on implicit roles, I posted a
> >         design
> >         doc here:
> >         *
> https://docs.google.com/document/d/1SCFfrBd4edSY3bVCMrNJYMxIVllD0bHJuGmgG-4vCXA/edit?usp=sharing*
> >         <
> https://docs.google.com/document/d/1SCFfrBd4edSY3bVCMrNJYMxIVllD0bHJuGmgG-4vCXA/edit?usp=sharing
> >
> >         -- any feedback on the design doc would be very welcome!
> >
> >         It seems we need to decide between implicit and dynamic roles --
> it
> >         wouldn't make sense to implement both.
> >
> >         I'd like to suggest that we implement implicit roles, rather than
> >         dynamic roles. The reason is simplicity: with implicit roles, we
> >         can
> >         simply remove the list of "legal" roles, and instead rely on ACLs
> >         and
> >         the authorization mechanism to decide whether operations like
> >         registering a framework as a role or making a reservation are
> >         allowed.
> >         By removing the list of roles, we have one less piece of state we
> >         need
> >         to account for, store in the replicated log, provide HTTP
> >         endpoints to
> >         modify, etc.
> >
> >         Let me know what you think!
> >
> >         Once we have implicit roles, we will naturally want to support
> (a)
> >         dynamic configuration of ACLs (b) dynamic configuration of
> >         weights. If
> >         you agree that implicit roles make sense, then I'd like to
> propose
> >         that I implement implicit roles, while you can focus on doing
> >         dynamic
> >         weights. After those are both built, we can decide how to build
> >         dynamic ACLs.
> >
> >         Neil
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message