hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4981) limit the max containers to assign to one application per heartbeart
Date Thu, 21 Apr 2016 21:32:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252787#comment-15252787
] 

Karthik Kambatla commented on YARN-4981:
----------------------------------------

Does limiting to one application per heartbeat solve this problem? What happens if the scheduler
assigns multiple CPU-intensive tasks (from different applications) on the same node? If the
job is CPU-intensive, may be it should ask for more vcores so the CPU is not oversubscribed?


Also, [~ywskycn] and I were looking into improving this in the past. We were considering determining
maxAssign dynamically based on the delta between the node's allocation and the overall average
allocation per node in the cluster. What do you think of that? 

> limit the max containers to assign to one application per heartbeart
> --------------------------------------------------------------------
>
>                 Key: YARN-4981
>                 URL: https://issues.apache.org/jira/browse/YARN-4981
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.5.0, 2.6.4
>            Reporter: ChenFolin
>         Attachments: YARN-4981.patch
>
>
> When use assignMultiple´╝îand such as set maxAssign=10:
> if a job is high CPU util or high CPU load, it may lead to  a part of nodes   very busy.
and the job may cost a long time.
> so , i want to limit the max containers to assign to one application per heartbeart,
it may help to nodes use more uniform.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message