mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <benjamin.mah...@gmail.com>
Subject Re: Quota Design Doc v1
Date Wed, 15 Jul 2015 22:02:17 GMT
If we omit authz for admin endpoints like this, it would be great to
disable them _by default_ until we have proper authorization in place.
Otherwise operators will accidentally leave admin endpoints open for anyone
to hit! :)

On Wed, Jul 15, 2015 at 12:07 PM, Alex Rukletsov <alex@mesosphere.com>
wrote:

> Folks,
>
> We have updated the design doc [1] based on numerous comments on the v1.
>
> There is one substantial design question left: what entity should decide
> whether a quota can be granted: the Master, an allocator, or a separate
> "Quota Manager". I have described pros and cons of each in the doc, please
> look inside for more information.
>
> Apart of that, I have made the following amendments:
> * Clarified the absence of authz and how this mitigate that with firewall;
> * Updated confusing naming: quota is a pair of guaranteed resource and
> limit;
> * Added a safety design principal;
> * Updated section explaining granting quota decision;
> * Updated QuotaInfo protobuf;
> * Added an alternative quota implementation via dynamic reservations;
> * Updated HTTP api to be more REST-like;
> * Added more ideas to the wip Allocator section.
>
> Please have a look and check whether your concerns have been addressed.
>
> [1]:
>
> https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I/edit#
>
> On Fri, Jul 10, 2015 at 8:52 AM, Jörg Schad <joerg@mesosphere.io> wrote:
>
> > I would  like to propose one alternative MVP proposal for the actual
> quota
> > implementation.
> >
> > Instead of making changes to the allocator we could have an allocator
> > agnostic “Quota manager” which builds on top the existing dynamic
> > reservations.
> >
> > Beyond MVP, we would still allow for allocator based implementations for
> > more complex quota mechanism, but this quota manager could also offer
> quota
> > support for allocators which are not aware of quota (TBD).
> >
> >
> > General Flow:
> >
> > - Quota manager leverages dynamic reservations to fulfill quota requests.
> > It basically continuously tries to match the desired state (quota
> requests)
> > with the actual state (dynamic reservations).
> >
> > - it receives quota requests from master
> >
> > - using (approximate) slave usage information it selects target
> slave/agent
> >
> > - requests to dynamic reservation (HTTP endpoint?), if fails retry
> >
> > - Master should notify QM of slave failures and reregistrations
> >
> > - in case of slave failures needs to request new reservation
> >
> > - in case of slave reregistrations might need to unreserve
> >
> > Required state of “Quota Manager”
> >
> >            - current dynamic reservations
> >
> >     - current granted quota
> >
> > - approximate slave available resources (in order to decide on which
> slave
> > to /reserve, but doesn’t matter if slightly out of sync -> in worst case
> > /reserve fails and we have to retry)
> >
> > Failover
> >
> >             - current granted quota are persisted in registry by master
> >
> > - current dynamic reservations are reconstructed from slave
> reregistration
> >
> > Master needs to propagate information about slave failures to Quota
> > Manager, as it might need to create new dynamic reservation in that case
> >
> >
> > Advantages compared to allocator based implementation:
> >
> > - no need to change allocator(s) for MVP
> >
> > - quota support for external quota-agnostic allocators
> >
> > - re-using existing mechanisms
> >
> > - minimal implementation effort for MVP
> >
> > - almost free support for quota chunks (even in MVP) as dynamic
> > reservations are per slave
> >
> > Disadvantages
> >
> > - allocator based quota implementation still needed for more elaborate
> > implementations
> >
> > - as dynamic reservations do not account towards fair share, so wouldn’t
> > quota based on this implementation. In my opinion this is not a real
> > problem as a) we did not really define the semantics of quota and b) fair
> > share is a allocator internal (i.e. DRF internal) notion so other
> allocator
> > implementations are free to do that differently anyhow.
> >
> > Looking forward to feedback!
> > Jörg
> >
> > On Thu, Jul 9, 2015 at 11:52 PM, Tomás Senart <tomas@mesosphere.io>
> wrote:
> >
> > > What about "Global Reservations"?
> > >
> > > On Thu, Jul 9, 2015 at 3:25 PM, Marco Massenzio <marco@mesosphere.io>
> > > wrote:
> > >
> > > > I've added my twocent in the doc - my vote goes for "Guaranteed
> > > Allocation"
> > > > - not as catchy as "Quota" (and will make classes' naming a
> challenge!)
> > > but
> > > > maybe more helpful in the long-term.
> > > >
> > > > Anyone has a better suggestion, please do... I can't really say I'm
> > > > super-excited by Guaranteed Allocation myself!
> > > >
> > > > *Marco Massenzio*
> > > > *Distributed Systems Engineer*
> > > >
> > > > On Thu, Jul 9, 2015 at 1:48 AM, Alex Rukletsov <alex@mesosphere.com>
> > > > wrote:
> > > >
> > > > > And you're not the only one who were confused by the terminology!
> One
> > > of
> > > > > the alternatives that didn't make it to the public doc was
> > > "cluster-wide
> > > > > dynamic reservations". The reason we preferred "quota" to " ...
> > > > > reservation" is because the latter is already overloaded with
> > meanings
> > > in
> > > > > Mesos world (static reservations, dynamic reservations). I have
> hoped
> > > the
> > > > > Terminology section would have helped to avoid the confusion, but
I
> > see
> > > > it
> > > > > doesn't. We'll think about how we can solve the problem, we
> > definitely
> > > > > don't want to create one more "libprocess process represented as
a
> > > thread
> > > > > in an OS process" ; ).
> > > > >
> > > > > I see your point regarding authorization, you're not alone here
> > either
> > > :
> > > > ).
> > > > > Some folks mentioned that the lack of authz is a blocker and will
> > > prevent
> > > > > them from upgrading the cluster. I would propose to treat MVP as
> > > > > experimental feature: use it at your own risk or disable endpoints
> > > > related
> > > > > to quota and hence the entire feature. Does it make sense?
> > > > >
> > > > > On Wed, Jul 8, 2015 at 7:10 PM, James Peach <jorgar@gmail.com>
> > wrote:
> > > > >
> > > > > >
> > > > > > > On Jul 4, 2015, at 3:15 AM, Alex Rukletsov <
> alex@mesosphere.com>
> > > > > wrote:
> > > > > > >
> > > > > > > Folks,
> > > > > > >
> > > > > > > Jörg and I are working on adding *quota* support to Mesos.
> Quota
> > > can
> > > > be
> > > > > > > described as cluster-wide dynamic reservation. I would
like to
> > > share
> > > > > the
> > > > > > > design doc [1] to gather community feedback early in the
design
> > > > phase.
> > > > > >
> > > > > > The most confusing part of this document to me was the 'quota'
> > > > > > terminology. Quotas normally refer to administrative limits
(esp.
> > > disk
> > > > > > quotas with hard and soft limits), not reserving resources.
Since
> > > what
> > > > > you
> > > > > > are describing is an extension to the resource reservation
> system,
> > it
> > > > > would
> > > > > > be clearer if it was described in those terms.
> > > > > >
> > > > > > I was also concerned that access control / authorization is
not
> > > planned
> > > > > > for the initial implementation. I think that if Mesos is to
have
> an
> > > > > > authorization policy, it should be applied uniformly following
> the
> > > > > > principle of least surprise.
> > > > > >
> > > > > > > The doc is work in progress, especially the part related
to
> quota
> > > > > support
> > > > > > > in the allocator. We think we can start working on adding
quota
> > > > support
> > > > > > to
> > > > > > > Mesos Master while fleshing out the design for how quota
is
> > handled
> > > > by
> > > > > > the
> > > > > > > built-in allocator.
> > > > > > >
> > > > > > > While working on the design, we faced some challenges and
> design
> > > > > > questions.
> > > > > > > One of them is what decisions should be deferred to allocator
> and
> > > > what
> > > > > > can
> > > > > > > be decided by the Master. We elaborate on this in the doc.
> > > > > > >
> > > > > > > Looking forward to your feedback!
> > > > > > >
> > > > > > > [1]:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I/edit?usp=sharing
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message