mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Mann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-8935) Quota limit "chopping" can lead to cpu-only and memory-only offers.
Date Thu, 21 Jun 2018 01:01:00 GMT

    [ https://issues.apache.org/jira/browse/MESOS-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16518769#comment-16518769
] 

Greg Mann commented on MESOS-8935:
----------------------------------

{code}
commit 79547292b58eafe1fbfacce4844533a677084bbe (HEAD -> master, origin/master, origin/HEAD,
meng/allocatable-flag)
Author: Meng Zhu <mzhu@mesosphere.io>
Date:   Wed Jun 20 17:00:18 2018 -0700

    Added a test to verify the `min_allocatable_resources` flag.

    This test verifies that the allocator honors the
    `min_allocatable_resources` flag and only offers resources
    that contain at least one of the specified resource quantities.

    Review: https://reviews.apache.org/r/67517/
{code}
{code}
commit a7e31366daa15fafd1e9aa21e81a04fadd961e37
Author: Meng Zhu <mzhu@mesosphere.io>
Date:   Wed Jun 20 17:00:03 2018 -0700

    Added a master flag to configure minimum allocatable resources.

    This patch adds a new master flag `min_allocatable_resources`.
    It specifies one or more resource quantities that define the
    minimum allocatable resources for the allocator. The allocator
    will only offer resources that contain at least one of the
    specified resource quantities.

    For example, the setting `disk:1000|cpus:1;mem:32` means that
    the allocator will only allocate resources when they contain
    1000MB of disk, or when they contain both 1 cpu and 32MB of
    memory.

    The default value for this new flag is such that it maintains
    previous default behavior.

    Also fixed all related tests and updated documentation.

    Review: https://reviews.apache.org/r/67513/
{code}
{code}
commit c3709616e75ab540b0bba45a5096e2d90fe6c133
Author: Meng Zhu <mzhu@mesosphere.io>
Date:   Wed Jun 20 16:59:58 2018 -0700

    Added a resource utility `isScalarQuantity`.

    `isScalarQuantity()` checks if a `Resources` object
    is a "pure" scalar quantity; i.e., its resources only have
    name, type (set to scalar) and scalar fields set.

    Also added tests.

    Review: https://reviews.apache.org/r/67516/
{code}
{code}
commit b0888195edf50fa8cf986525300dfd4f9420d812
Author: Meng Zhu <mzhu@mesosphere.io>
Date:   Wed Jun 20 16:59:54 2018 -0700

    Fixed a bug in `createStrippedScalarQuantity()`.

    This patch fixes `createStrippedScalarQuantity()` by
    stripping the revocable field in resources.

    Also added a test.

    Review: https://reviews.apache.org/r/67510/
{code}

> Quota limit "chopping" can lead to cpu-only and memory-only offers.
> -------------------------------------------------------------------
>
>                 Key: MESOS-8935
>                 URL: https://issues.apache.org/jira/browse/MESOS-8935
>             Project: Mesos
>          Issue Type: Bug
>          Components: allocation
>    Affects Versions: 1.4.2, 1.5.0, 1.5.1, 1.6.0
>            Reporter: Benjamin Mahler
>            Assignee: Meng Zhu
>            Priority: Major
>
> When we allocate resources to a role, we'll "chop" the available resources of the agent
up to the quota limit for the role (per MESOS-7099). This prevents the role from exceeding
its quota limit.
> This has the unintended consequence of creating cpu-only and memory-only offers.
> Consider agents with 10 cpus and 100 GB mem and roles with quota guarantee/limit of 5
cpus, 10 GB mem. The following allocations will occur:
> agent 1:
>  r1 -> 5 cpus 10GB mem
>  r2 -> 5 cpus 10GB mem
>  r3 -> 0 cpus 10GB mem (quota allocates even if it can make progress towards a single
resource and MESOS-1688 allows this)
>  r4 -> 0 cpus 10GB mem
>  ...
>  r10 -> 0 cpus 10GB mem
> agent 2:
>  r3 -> 5 cpus 0GB mem (r3 is already at its 10GB mem limit)
>  r4 -> 5 cpus 0GB mem
>  r11 -> 0 cpus 10GB mem
>  ...
>  r20 -> 0 cpus 10GB mem
> Here, roles 3-20 receive memory only and cpu only offers. This gets further exacerbated
if DRF chooses the same ordering between roles across cycles. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message