mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Rukletsov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-1718) Command executor can overcommit the slave.
Date Thu, 08 Dec 2016 21:40:59 GMT

    [ https://issues.apache.org/jira/browse/MESOS-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733469#comment-15733469
] 

Alexander Rukletsov commented on MESOS-1718:
--------------------------------------------

 If {{--cgroups_enable_cfs}} is enabled, the overcommit because of unaccounted executor CPU
resources may effectively defeat the CFS limits, depeniding on the ratio of task's CPU to
executor's.

Consider the following scenario. A framework attempts to run numerous small command-based
computationally intensive tasks with {{0.1}} CPU requirement each. Since the allocator does
not account extra resources, it is possible to schedule {{10}} such tasks per physical CPU
on one agent. However those extra resource do increase the CFS quota, which is effectively
{{0.2 × CFS period}} per task. I would argue this can be surprising and undesirable in some
cases.

> Command executor can overcommit the slave.
> ------------------------------------------
>
>                 Key: MESOS-1718
>                 URL: https://issues.apache.org/jira/browse/MESOS-1718
>             Project: Mesos
>          Issue Type: Bug
>          Components: agent
>            Reporter: Benjamin Mahler
>
> Currently we give a small amount of resources to the command executor, in addition to
resources used by the command task:
> https://github.com/apache/mesos/blob/0.20.0-rc1/src/slave/slave.cpp#L2448
> {code: title=}
> ExecutorInfo Slave::getExecutorInfo(
>     const FrameworkID& frameworkId,
>     const TaskInfo& task)
> {
>   ...
>     // Add an allowance for the command executor. This does lead to a
>     // small overcommit of resources.
>     executor.mutable_resources()->MergeFrom(
>         Resources::parse(
>           "cpus:" + stringify(DEFAULT_EXECUTOR_CPUS) + ";" +
>           "mem:" + stringify(DEFAULT_EXECUTOR_MEM.megabytes())).get());
>   ...
> }
> {code}
> This leads to an overcommit of the slave. Ideally, for command tasks we can "transfer"
all of the task resources to the executor at the slave / isolation level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message