mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Qian Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-4027) Improve task-node affinity
Date Sat, 26 Dec 2015 12:37:49 GMT

    [ https://issues.apache.org/jira/browse/MESOS-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15071874#comment-15071874
] 

Qian Zhang commented on MESOS-4027:
-----------------------------------

After more consideration, I think there is a potential blocking issue for implementing this
feature: Currently in Mesos, each framework can only be offered a subset of Mesos cluster
resources, i.e., framework does not have a global view of the whole Mesos cluster (can only
see some Mesos agents rather than all). This might be OK for framework which needs task-to-node
anti-affinity, but seems not OK for task-to-node affinity since there may be an existing task
running in a Mesos agent, but it is possible that Mesos agent will not be offered to the framework
which wants to launch a new task in the same node with that existing task. This issue should
be gone once we introduce [Optimistic Offer|https://issues.apache.org/jira/browse/MESOS-1607]
because with optimistic offer each framework will have a global view for the whole Mesos cluster.

> Improve task-node affinity
> --------------------------
>
>                 Key: MESOS-4027
>                 URL: https://issues.apache.org/jira/browse/MESOS-4027
>             Project: Mesos
>          Issue Type: Wish
>          Components: allocation, general
>            Reporter: Chris
>            Assignee: Qian Zhang
>            Priority: Trivial
>
> Improve task-to-node affinity and anti-affinity (running hadoop or spark jobs on a node
currently running hdfs or to avoid running Ceph on HDFS nodes).
> Provide a user-mutable Attribute in TaskInfo (the Attribute is modified by a Framework
Scheduler) that can describe what a Task is running.
> The Attribute would propagate to a Task at execution. The Attribute is  passed to Framework
Schedulers as part of an Offer's Attributes list. 
> A Framework Scheduler could then filter out or accept Offers from Nodes that are currently
labeled with a desired set or individual Attribute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message