mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: [Discussion] MESOS-4442: `allocated` may have more resources then `total` in allocator
Date Fri, 29 Jan 2016 21:04:07 GMT
Looks like you've seen the comment here:
https://issues.apache.org/jira/browse/MESOS-4442?focusedCommentId=15120754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15120754

We allow this so that the allocator can be over-allocated. When
over-allocated, we will not re-offer freed resources until we are no longer
over-allocated.

On Fri, Jan 22, 2016 at 4:36 PM, Klaus Ma <klaus1982.cn@gmail.com> wrote:

> @team, any comments?
>
> ----
> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> Platform OpenSource Technology, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Thu, Jan 21, 2016 at 9:31 PM, Klaus Ma <klaus1982.cn@gmail.com> wrote:
>
> > Yes, *total*: cpus(*):2 vs. *allocated*: cpus(*):2;cpus(*){REV}:2
> >
> > ----
> > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > Platform OpenSource Technology, STG, IBM GCG
> > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> >
> > On Thu, Jan 21, 2016 at 5:43 PM, Qian Zhang <zhq527725@gmail.com> wrote:
> >
> >> In the log you posted, it seems total cpus is also 2 rather than 1, but
> it
> >> seem there are 4 allocated cpus (2 non-revocable and 2 revocable)?
> >>
> >> I0121 17:08:09.303431 4284416 hierarchical.cpp:528] Slave
> >> f2d8b550-ed52-44a4-a35a-1fff81d41391-S0 (9.181.90.153) updated with
> >> oversubscribed resources  (total: cpus(*):2; mem(*):1024; disk(*):1024;
> >> ports(*):[31000-32000], allocated: cpus(*):2; mem(*):1024; disk(*):1024;
> >> ports(*):[31000-32000]; *cpus(*){REV}:2*)
> >>
> >>
> >> Thanks,
> >> Qian Zhang
> >>
> >> On Thu, Jan 21, 2016 at 5:25 PM, Klaus Ma <klaus1982.cn@gmail.com>
> wrote:
> >>
> >> > Hi team,
> >> >
> >> > When I double-check the feature interaction between Optimistic Offer
> >> Phase
> >> > 1 & Oversubscription, I found an issue that `allocated` may have more
> >> > resources then `total` in allocator when enable Oversubscription. I'd
> >> like
> >> > to get your input on whether this is design behaviour, although the
> >> impact
> >> > is low: 1.) allocator will not offer this delta resources, 2) QoS
> >> > Controller will correct it later by killing the executor. Personally,
> >> I'd
> >> > like to keep this assumption in allocator: slave.total always contains
> >> > slave.allocated.
> >> >
> >> > Here's the steps:
> >> >
> >> > T1: in cluster, cpus=2: one is revocable and the other one is
> >> nonRevocable
> >> > T2: framework1 get offer cpus=2, launch task but estimator report
> empty
> >> > resources before executor launched
> >> > T3: slave.total is updated to cpus=1 in
> >> > HierarchicalAllocatorProcess::updateSlave
> >> > T4: in allocate(), slave.total (cpus=1) < slave.allocated (cpus=2)
> >> >
> >> > Here's the log I got:
> >> >
> >> > I0121 17:08:09.303431 4284416 hierarchical.cpp:528] Slave
> >> > f2d8b550-ed52-44a4-a35a-1fff81d41391-S0 (9.181.90.153) updated with
> >> > oversubscribed resources  (total: cpus(*):2; mem(*):1024;
> disk(*):1024;
> >> > ports(*):[31000-32000], allocated: cpus(*):2; mem(*):1024;
> disk(*):1024;
> >> > ports(*):[31000-32000]; *cpus(*){REV}:2*)
> >> >
> >> > Please refer to MESOS-4442 for more detail.
> >> >
> >> > ----
> >> > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> >> > Platform OpenSource Technology, STG, IBM GCG
> >> > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message