mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jie Yu <yujie....@gmail.com>
Subject Re: Proposing change to the allocatable check in the allocator
Date Tue, 12 Jun 2018 05:36:23 GMT
I would suggest we also consider the possibility of adding per framework
control on `min_allocatable_resources`.

If we want to consider supporting per-framework setting, we should probably
model this as a protobuf, rather than a free form JSON. The same protobuf
can be reused for both master flag, framework API, or even supporting
Resource Request in the future. Something like the following:

message ResourceQuantityPredicate {
  enum Type {
    SCALAR_GE,
  }
  optional Type type;
  optional Value.Scalar scalar;
}
message ResourceRequirement {
  required string resource_name;
  oneof predicates {
    ResourceQuantityPredicate quantity;
  }
}
message ResourceRequirementList {
  // All requirements MUST be met.
  repeated ResourceRequirement requirements;
}

// Resource request API.
message Request {
  repeated ResoruceRequrementList accepted;
}

// `allocatable()`
message MinimalAllocatableResources {
  repeated ResoruceRequrementList accepted;
}

On Mon, Jun 11, 2018 at 3:47 PM, Meng Zhu <mzhu@mesosphere.com> wrote:

> Hi:
>
> The allocatable
> <https://github.com/apache/mesos/blob/1.5.x/src/master/allocator/mesos/hierarchical.cpp#L2471-L2479>
>  check in the allocator (shown below) was originally introduced to
>
> help alleviate the situation where a framework receives some resources,
> but no
>
> cpu/memory, thus cannot launch a task.
>
>
> constexpr double MIN_CPUS = 0.01;constexpr Bytes MIN_MEM = Megabytes(32);
> bool HierarchicalAllocatorProcess::allocatable(
>     const Resources& resources)
> {
>   Option<double> cpus = resources.cpus();
>   Option<Bytes> mem = resources.mem();
>
>   return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
>          (mem.isSome() && mem.get() >= MIN_MEM);
> }
>
>
> Issues
>
> However, there has been a couple of issues surfacing lately surrounding
> the check.
>
>    -
>    - - MESOS-8935 Quota limit "chopping" can lead to cpu-only and
>    memory-only offers.
>
> We introduced fined-grained quota-allocation (MESOS-7099) in Mesos 1.5.
> When we
>
> allocate resources to a role, we'll "chop" the available resources of the
> agent up to the
>
> quota limit for the role. However, this has the unintended consequence of
> creating
>
> cpu-only and memory-only offers, even though there might be other agents
> with both
>
> cpu and memory resources available in the cluster.
>
>
> - MESOS-8626 The 'allocatable' check in the allocator is problematic with
> multi-role frameworks.
>
> Consider roleA reserved cpu/memory on an agent and roleB reserved disk on
> the same agent.
>
> A framework under both roleA and roleB will not be able to get the
> reserved disk due to the
>
> allocatable check. With the introduction of resource providers, the
> similar situation will
>
> become more common.
>
> Proposed change
>
> Instead of hardcoding a one-size-fits-all value in Mesos, we are proposing
> to add a new master flag
>
> min_allocatable_resources. It specifies one or more scalar resources
> quantities that define the
>
> minimum allocatable resources for the allocator. The allocator will only
> offer resources that are more
>
> than at least one of the specified resources.  The default behavior *is
> backward compatible* i.e.
>
> by default, the flag is set to “cpus:0.01|mem:32”.
>
> Usage
>
> The flag takes in either a simple text of resource(s) delimited by a bar
> (|) or a JSON array of JSON
>
> formatted resources. Note, the input should be “pure” scalar quantities
> i.e. the specified resource(s)
>
> should only have name, type (set to scalar) and scalar fields set.
>
>
> Examples:
>
>    - - To eliminate cpu or memory only offer due to the quota chopping,
>    - we could set the flag to “cpus:0.01;mem:32”
>    -
>    - - To enable offering disk only offer, we could set the flag to
>    “disk:32”
>    -
>    - - For both, we could set the flag to “cpus:0.01;mem:32|disk:32”.
>    - Then the allocator will only offer resources that at least contain
>    “cpus:0.01;mem:32”
>    - OR resources that at least contain “disk:32”.
>
>
> Let me know what you think! Thanks!
>
>
> -Meng
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message